5.4 Finding meaning in the data

Overview   Introduction   Statistical Assessment  Mechanics of Interpretation   Finding Meaning in the Data  Results and Next Steps

In almost every case, the goal for an expensive study will involve an implied question. Has an expensive fish bypass structure improved salmon passage? Has a new regulation resulted in more fish passed to the spawning ground? Has the stock continued to decline after an expensive restriction on land use? These are not questions that can be answered by either a table full of numbers, nor by a simple statistical hypothesis test. These are questions that can only be answered, often imperfectly, by a careful study of the history of the problem, a study of what has occurred in similar situations elsewhere, and by a study of the current statistical indicators in the context of a larger times series of these indicators. Also, answering these questions, again sometimes imperfectly, requires the use of principles and accepted assumptions that come from other fields of study, such as physics, engineering, or hydrology. In short, interpreting fishery data should be difficult and time consuming, or else it is not being done correctly. 

Of course, a full interpretation of the data has very little in common with advocacy or politics, although many times there will be strong political pressure for a scientific analyst to push the interpretation one way or the other. In its pure form, the scientific problem is simply about understanding the underlying processes. In the 1960s scientists brought forward

Enlarge Image

Reading otoliths to detect hatchery-produced salmon in Prince William Sound, Alaska. Credit Hal Geiger/ADF&G.

Reading otoliths to detect hatchery-produced salmon in Prince William Sound, Alaska. Credit Hal Geiger/ADF&G.

information about the effects of cigarette smoking on health. Actually, the question of whether or not someone should smoke is not a scientific question -- but predicting and explaining the likely effects of cigarette smoking is. This example of cigarette smoking should provide the model for how fisheries science should proceed. The job of science is not to tell people what to do, but part of its job is is to help people understand what will happen if we do certain things. These conclusions must be based on a logical series of steps. An important fact about the craft of science is that it is never good enough to just have the right answer. The craft is all about showing the reader the necessary assumptions and principles that lead to the right answer, supporting the conclusions in a logical and orderly way, and then providing a reasonable and understandable qualification of the answer, as needed.  

To look at a simple example, a study might be put in place to answer the question is a particular hatchery contributing fish to a particular fishery? A study might involve the use of coded-wire tags. At the end of the study, the analyst could develop confidence intervals, and these confidence intervals might be very small -- seeming to imply a high degree of statistical confidence. Yet, if there is a chance that some samplers overlooked some of the these tags that they were expecting to find in a sampling program, then this is an important part of the analysis that the analyst must bring to the interpretation, as part of a complete look at the larger question. Explaining this source of uncertainty will be much harder than just computing a confidence interval -- but this source of uncertainty may actually be much more important. Interpretation involves a synthesis of all of the relevant information. Again, in the end, the analyst may need to provide some responsible speculation or provide some additional untested assumptions. The analyst will need to bring in information collected outside of the study. A well crafted study will give decision makers the best information possible when they address a decision how to use the fish hatchery. The question of whether the hatchery is good or bad, or whether it should be closed or receive additional funding are not scientific questions. These questions are value judgments and public-policy issues. The most important test of the quality of the interpretation step is is whether decision makers and the public find understandable and credible guidance about the probable effects of decisions that are under consideration.