1.2.2 Mechanisms: Basic questions

A cautionary note about confounding of interpretations about mechanisms

 

We begin with a cautionary note so that you do not have unrealistic expectations about identifying mechanisms. The dynamics of any ecosystem, including those containing salmon, are affected by multiple simultaneously operating natural and human sources of variability. The result is that interpretations of data collected from salmon systems will be confounded; that is, some observed change or trend may be the result of more than one mechanism. Because of this confounding, it is unlikely that definitive evidence of causes of changes in salmon indicators will emerge from non-experimental, observational monitoring programs, which are the types of programs dealt with on this web site. Instead, definitive evidence will require direct, human-controlled, manipulative experiments. Unfortunately, such experiments are generally not feasible for wild Pacific salmon, and so your expectations about understanding mechanisms should be adjusted accordingly. Such experiments would require deliberate manipulation of potential causal factors (the explanatory variables) in randomly chosen "treated" experimental units (e.g., areas and salmon populations exposed to a given level and type of climatic change). The salmon response variables or indicators would be observed there and then compared with observations in randomly assigned "control" or contrasting treatment units (those exposed to different levels of that climatic change or none at all). Obviously, such large-scale experiments are impractical on the scale required to understand climatic mechanisms.

Partial solutions to confounding of interpretations about mechanisms

 

In certain situations, carefully designed and implemented passive, non-experimental monitoring programs can reduce the chance of creating inadvertent biases in estimates of salmon indicators. Such programs can increase the chance of reliably estimating the relative contribution of different mechanisms to changes in salmon indicators. Such designs are the subject of this section. For instance, imagine that climatic change has reduced low-summer water flows over the last decade to a greater extent in interior streams than in coastal streams. If all else is identical for these interior and coastal habitats, including their salmon populations and human disturbances in those habitats, then such spatial differences in climatic influence should be discernible by comparing freshwater survival rates of salmon in the two regions. Of course, all else is never identical, but there are advanced designs that can lead to some reasonably rigorous conclusions under certain circumstances. You will be able to see those designs later in a section of Step 2 on designing monitoring programs.

One of the biggest challenges with trying to understand effects of climate on salmon is that climatic effects are likely to extend over large spatial areas, rather than just being localized. This situation reduces the opportunities for comparing cases with differing levels of climatic influence. For this reason, one of the more practical sources of evidence regarding effects of climate change on salmon will be from cases in which there is a natural gradient across populations in those effects.

Remember, though, that passive observational monitoring programs still cannot achieve the same high level of confidence for or against hypothesized mechanisms as active, manipulative experiments, where various factors and disturbances might be under more human control. For these same reasons, it is not appropriate to extrapolate findings in one situation to predict future effects of some causal mechanism in situations other than those surveyed. This limitation on the "universe of inference" is expanded upon in questions below.

Nevertheless, some conclusions about causal mechanisms behind observed changes can be derived from careful analysis and comparisons among situations. An excellent example of this is illustrated by the The United States Environmental Protection Agency's web site on "Causal Analysis/Diagnosis Decision Information System (CADDIS)". CADDIS is an online application that helps scientists and engineers in the United States to "find, access, organize, use, and share information to conduct causal evaluations in aquatic systems." Like this mechanisms-oriented section of the Salmon Monitoring Advisor web site that you are currently using, the CADDIS site helps guide users through a series of steps to rigorously collect, organize, and analyze data. 

 

Indirect methods for estimating strength of evidence for different mechanisms

 

As noted above, we are not able to conduct human-controlled large-scale experiments on wild Pacific salmon to identify the relative importance of climatic mechanisms compared to all other causes of changes in attributes of salmon. Therefore, we must both reduce expectations about finding definitive evidence of climatic change on salmon and rely on more indirect means of deducing such effects. Such indirect methods have been used in many fields. For instance, early work in medical statistics (Hill 1965 and Hill 1971, as reviewed by Stewart-Oaten 1996) identified several types of evidence that can contribute to increased confidence about interpretations of causal mechanisms. These types of evidence apply equally well to our salmon cases in which data will be collected from passive observational (i.e., non-experimental) monitoring programs.

1. How plausible is the hypothesized causal mechanism? Based on known physical and biological principles, is the proposed mechanism realistic?
   
2. What is the strength of the estimated effect? The stronger it is, the more likely we are to correctly distinguish the mechanism causing an observed response from background variation and observation error, as well as separate it from changes arising from other simultaneously operating mechanisms. Note that in such analyses, emphasis is on estimating the strength of some effect, rather than on formally testing some hypothesis about the mechanism having an effect or no effect. For more information on estimation of effect sizes, see the next expandable text box on "Focus on parameter estimation". 
   
3. The consistency of direction, magnitude, and duration of observed effects across studies of similar systems also lends credibility to a hypothesis about a given mechanism causing those effects. For instance, does empirical evidence show such a mechanism working in the same way for other species or situations?
4. The specificity of effects should match the proposed mechanism such that, for instance, species or life stages that should not be affected by the mechanism do not show change, whereas the stages that should be affected do show a response. 
   
5. The timing of observed changes should occur simultaneously with, or after, the change in the state variable of the proposed causal mechanism. If there is a time lag in the response, it should be on a realistic time scale based on what is known about the processes involved.
   
6. Similar to lines of evidence #4 and #5, observed changes along a physical or biological gradient should be related to the exposure or strength of the purported causal mechanism at those locations. 
   
7. Also, similarity or coherence of responses across space, time, populations, species, and indicators can strengthen the case for a particular mechanism. 
   
8. "Natural experiments" or at least contrasting dynamic behavior at different times or places, are excellent potential sources of fortuitously created comparison groups. These are not human-manipulated experiments, but they may be similar enough to learn about mechanisms causing observed changes.  
        Here are two examples, including one from salmon, to illustrate "natural experiments". Winter flooding events of different magnitudes can scour eggs from gravels in some areas more than in others, creating a natural set of "contrasting treatment" areas that could be sampled and compared. A famous non-salmon example is the eruption of Mt. Pinatubo in 1991, which created an important line of evidence regarding the global warming debate. That eruption and the resulting injection of material into the atmosphere created substantially cooler temperatures afterwards due to the increased reflection of sunlight back into space.  If these types of "natural experiment" situations are appropriately monitored (for example, with before/after (BA), before/after/control/impact (BACI), or enhanced BACI designs such as BACIPs (BACI with paired series)), they can build evidence for or against particular hypothesized mechanisms. However, there are several potential problems of interpretation with the first two designs (BA and BACI). For more information on these and other types of designs, as well as limitations on extrapolating their results to other situations, see Underwood (1994) and Schwarz (1998). We also discuss the pros and cons of these designs later in Step 2.
   

 

Focus on parameter estimation

 

Many ecologists as well as statisticians argue convincingly that researchers should focus more on estimating effect sizes of factors of interest and uncertainties in those estimates, rather than formally testing hypotheses with standard inference methods that involve calculating P values (Edwards 1992; Stewart-Oaten 1996). We agree that the purpose of designing monitoring programs for understanding mechanisms is not to formally test whether some hypothesized mechanism is statistically significant. In part, this is because with a large enough sample size, almost anything can be shown to be statistically "significant". Instead, what matters most is the estimated magnitude and direction of the effect. Salmon-climate researchers are trying to estimate the relative importance of different mechanisms, i.e., their contribution to the observed variation in a given salmon indicator. The key focus here then is on estimating those effects and providing confidence or probability intervals around those estimates. From that information, we can then say something about the relative importance of climatic change compared to human disturbances such as habitat degradation in freshwater or near-shore environments.

Categories of sampling situations

 

It is useful to conceptualize the main types of sampling situations in terms of how close they are to permitting an optimal design that can deduce causal mechanisms almost as well as a human-manipulated experiment. The key is to be able to differentiate as much as possible between natural spatial and/or temporal variation in salmon indicators and variation in them created by the mechanism of interest (climatic change here), and without confusing the latter with variation caused by human activities.

Roger Green's (1979) book on sampling for environmental biologists categorized various types of sampling programs for impact assessment. Since that time, several types of sampling designs have been developed to help differentiate between alternative hypotheses about causes of the observed changes in certain places and times. These designs will be useful to you when planning passive monitoring programs (as opposed to deliberate manipulative experiments) to estimate the relative importance of different mechanisms affecting salmon, including climatic change. Hicks et al. (1991) identified two major classes of designs, (1) those in which data are available "before" some change occurs in a possible driving mechanism, which permits comparison with the "after" situation, and (2) designs in which there is only one sampled unit or multiple units that represent some degree of replication. Some designs are relatively simple comparisons of situations (so-called Before-After, or BA designs) that are logistically relatively easy and inexpensive to implement (Green 1979). However, results from such simple designs cannot be extrapolated with confidence to other unsampled situations. Other designs are more complex (such as multiple spatial and temporal sampling situations to extend basic Before-After Control-Impact, or BACI designs) (e.g., Stewart-Oaten et al. 1986; Walters et al. 1988; Underwood 1994; Downes et al. 2002). We provide more discussion of these designs in Step 2.

This entire web site was originally developed in the context of concerns about the influence of climatic changes on wild salmon, so our examples of mechanisms reflect that focus. Some material in this section on mechanisms has been adapted from literature sources, including Schmitt and Osenberg (1996) and Schwarz (1998). Further details are found in those documents as well as others such as Green (1979) and NRC (1990).  

The indicators of wild Pacific salmon populations on which we focus are abundance, productivity, and diversity. These are the same indicators as described in the section parallel to this one called "Status and trend basic questions". In fact, this section on basic questions related to causal mechanisms is quite similar to "Status and trend basic questions", except that additional material here deals with our different purpose of understanding mechanisms underlying observed trends and status of salmon populations. Some salmon indicators such as productivity (e.g., adults produced per spawner), involve the entire life history of salmon. Mechanisms causing changes in such indicators could mainly occur in fresh water, the ocean, or both. Therefore, it is essential to produce reliable estimates of abundance of salmon for at least two life stages, for instance, the spawner-to-smolt and smolt-to-adult stages. The relative importance of freshwater vs. marine mechanisms can only be deduced with such data covering each life stage.

Mechanisms affecting salmon indicators include both (a) natural processes, such as variation in food and predator conditions encountered by juveniles in fresh water as well as the ocean, and (b) human activities, such as fish farms that produce diseases and sea lice or road construction that causes silt to clog gravels containing incubating salmon eggs. In ideal situations, we hope to use sampling designs that estimate effects of particular human activities of interest separately from effects of other human actions and natural processes. Results from such designs will increase our degree of belief in the relative importance of those different factors, including climatic change, and will lead to appropriately focused management actions and research priorities.

More information

 

To understand the relative importance of different mechanisms, it is essential that data be gathered not only on salmon, but also on the status and trend of habitat conditions. Those conditions include physical and biological indicators in the marine as well as freshwater environments. There are standard practices for collecting such habitat data (Downes et al. 2002; Roni 2005). Indicators of habitat condition may vary across species and life stages. Some examples are physical measures such as suspended sediments, concentration of contaminants, water temperature and stream flow, volume of cold-water refuge zone in lakes for sockeye juveniles, and amount of salmon spawning and rearing areas. Biological indicators of habitat condition include measures such as productivity of lakes and coastal oceans.

Indicators are the summary statistics on which management decisions are made.  The four basic indicators of salmon that are usually considered in the context of designing biological monitoring programs for salmon are: (1) abundance (or number of fish at some life stage); (2) productivity (or the survival rate from one life stage to another); (3) diversity (or variability within and among populations in life history types, age structure, mean date of migration, etc.); and (4) spatial structure (or the concentration of fish within parts of a specified area). The latter two categories are indirect indicators of resilience.

As noted above, monitoring indicators of status and trend in the quantity and quality of habitat (as opposed to salmon populations) is also required to evaluate hypotheses about potential causes of changes in salmon. However, habitat indicators are generally much more diverse and less standardized than indicators for status and trend monitoring of salmon. Both freshwater and marine habitat conditions influence salmon populations, so where possible, information is needed for both aquatic systems to help understand causes of changes in salmon.  As described in Step 3 - Collect data, researchers have documented substantial effects of several types of environmental variables on salmon, ranging from low freshwater flows in summers, gravel-scouring events from high flows, and high summer water temperatures, to ocean surface temperatures when juvenile salmon enter salt water, timing of the spring bloom, and other oceanographic factors.  This web site does not go into habitat monitoring in detail, in part because of the situation-specific and indicator-specific nature of advice about monitoring such variables. Nevertheless, many principles about monitoring salmon that are on this web site also apply to monitoring habitats and other parts of aquatic ecosystems. Readers can learn more about habitat indicators by looking at some reviews plus Nelitz et al. (2008) and Stalberg et al. (2009).

An obvious first step in designing a monitoring program is to indicate the species that will be monitored because that influences your spatial design, temporal design, and response design.  As an example, consider the differences between designing monitoring programs for fall Chinook salmon and coho salmon.  The spatial design may differ for the two species because fall Chinook tend to spawn in larger streams and rivers lower in watersheds, whereas coho tend to spawn in smaller tributaries higher in the watersheds.  This difference in sampling frame may mean that you can conduct a census of all areas where fall Chinook may be found, but you may have to conduct a survey of a subset of potential areas where coho may be found.  The temporal design may differ between the two species because of differences in run timing or length of residence time in your sample frame.  Response design may also differ between the two species because, for example, you may be able to conduct a count of live coho spawning in their smaller tributary streams, whereas you may not be able to see fall Chinook spawning in some of the larger, deeper rivers. 

Specifying the life stages of species you intend to monitor dictates where, how often, and how you need to sample.  For example, if the life-stage of interest is the number of downstream-migrating juvenile fish (e.g., smolts) you may be able to conduct a census of the population by operating a trap 24 hours a day in the spring at one location low in your sample frame.  If, however, your life-stage of interest is juveniles that rear throughout your sample frame, you will need to sample at more than one location at different times of year.

More information

 

It should always be clear to people who implement monitoring programs just who will ultimately use the resulting data and for what purposes. To avoid designing an inadequate monitoring program, the different needs of multiple groups of data users should be taken into account in terms of what they need monitored, as well as where and when they need it.

More information

 

A clear understanding of the reasons why that monitoring information is needed will assist you in developing many aspects of your monitoring program. The statistical rigor of information that is needed to evaluate cause-and-effect relationships (i.e., mechanisms) is substantially higher than what is needed to merely assess status and trends in salmon populations. Thus, it is useful for you to describe below why the information is needed in terms of how managers and other decision makers will use it. This will help you identify which specific monitoring information is needed.

More information

 

An important consideration for evaluating the efficacy of information provided by any monitoring program is the degree of confidence you have in the information and the likelihood that it will lead you to wrong conclusions.  In general, the more certain you need to be about the information you are providing, the more resources you will need to gather it. Thus, you should specify the desired precision and/or statistical power of your monitoring program early in its development. This will help determine the most appropriate monitoring design for your situation, given your available resources. Precision can be described in terms of the coefficient of variation (CV), which is the standard deviation (SD) of the estimates divided by the mean estimate. Depending on the quantity estimated, typical CVs for salmon variables or parameters range from 0.3 to over 1, i.e., a SD about 30% of the mean to at least as large as the mean value. Small CVs reflect greater precision.

More information

 

In order to adequately assess the resources needed to implement your monitoring program, it is important that you have a clear understanding of the total time span over which information will be gathered and the time intervals over which results are to be reported. These are essential first steps in developing the temporal design of your monitoring program.

More information

 

Similarly, you need to specify key spatial aspects of your monitoring program, which include the total geographic extent and discrete spatial resolution over which sampling will be done. For instance, will the extent be sampling across three watersheds or just one, and will samples be taken at every 100 m reach of streams within each watershed or only at a few randomly chosen spots?

Separate from these spatial aspects of the sampling scheme is the spatial extent across which people will wish to make inferences based on the information gathered by your monitoring program. For instance, if the relative importance of some causal mechanism is estimated in one region, will decision makers only be using that information to decide on mitigative actions for that sampling unit, in which case the "universe of inference" is very limited? Alternatively, will they be extrapolating those results to other salmon populations/regions to take appropriate mitigative actions there too? If the answer is yes to the last question, then this will require much more elaborate designs and situations in the field to help eliminate alternative explanations by providing appropriate sample groups for comparison (see Schwarz 1998 for more information on the limited universe of inference that results from most observational monitoring programs, and how to deal with this situation).

Thus, this information on where the monitoring information is needed will assist you in determining the survey design that is most appropriate, as well as the funding, equipment, and personnel that you will need to implement the monitoring program.

More Information

 

Specifying your target statistical populations is a vital step in developing the spatial design of your monitoring program.  A statistical population is where data for your metrics will be, or can potentially be, gathered.  The spatial domain of your statistical population can be continuous (as with point locations along a stream network or within a lake) or discrete (in terms of a set of defined reaches that describe the biological population’s domain).   We sample the statistical population, i.e., we select locations where we will make measurements, then make inferences about the biological population based on the site measurements.  The sites are usually located in the biological population's domain (we give an exception later).  For example, your biological population might be fish that spawn in a particular river.  Your metric for monitoring this population might be the number of redds that these fish make while spawning.  In this case your statistical population would be located within the biological domain (i.e. spawning population).  On the other hand, your metric for monitoring this population might be counts of fish as they pass upstream over a weir that is located downstream of spawning areas.  In this case your statistical population would be outside of the domain of your biological population. To see more information on comparisons of biological and statistical populations ...click here.

More information

 

Metrics are calculated from the measurements or observations that you make while implementing your sampling protocols.  For example, your sampling protocol might involve counting the number of live fish observed on the spawning grounds at regular intervals over the course of the spawning season. Your metric for these “measurements” might be to calculate an area-under-the-curve estimate of the total number of live spawners.  Metrics can themselves be equivalent to the indicators you previously identified, or they may be an intermediate step between field measurements/observations and your indicators of interest.  For example, if your indicator is the abundance of spawners in a particular river, and if you can count all fish that enter the river to spawn as they pass over a counting weir, then your metric (the total number of fish counted passing the weir) would be equivalent to your target indicator.  On the other hand, if you can only assess spawner abundance by counting spawning fish at a sub-sample of all the potential spawning sites, then your metrics (the average number of spawning fish observed/length of spawning habitat surveyed) might be an intermediate step to developing your target indicator of total abundance.  

Identifying the metrics you will be collecting is an important component of developing your response design. The frequency with which you will need to collect information on your metrics has an important influence on the development of your temporal and response design.  It also can influence your spatial design.  For example, if you have a metric that requires frequent data collection over the course of your reporting interval, then due to budget limitations you may need to choose a design that allows you to sample at fewer sites than a design for metrics for which data can be collected less frequently.

 

We often hear that variability affects our ability to describe the metrics for which we’ve designed our monitoring programs.  However, the term "variability" connotes so much that before we can make any progress evaluating how variability affects estimation, we need to describe what we mean by “variability”.  When we consider a monitoring program whose goals include describing ecological resources that are distributed across regions across years, and for which our measurements are imperfect, organizing variability helps us focus our attention on those aspects of variation that are of specific interest, and those that impose imprecision or bias on our descriptions.  The following is one way of decomposing variability to assist us in evaluating which components are important and which are distractions (that might be amendable to management by appropriate monitoring designs). The context of this brief overview is an interest in the status and trends of metrics among sites across years; think of a 'sites-by-years' matrix of metric scores for which status and trends information is desired.Consult the following for additional insights: Dauwalter, et al., 2009; Urquhart, et al., 1998; Urquhart and Kincaid, 1999; Wagner, et al., 2007; Wiley, et al., 1997.

Components of Variability

You will need information on four broad categories of variability that will be useful for designing your monitoring program.  The information might be available from existing studies or you might design your monitoring program to obtain the necessary data during its first few years.  The following text uses "year" as the temporal unit for ease of discussion and because readers are likely familiar with programs that use "year" as the primary temporal unit. 

• Spatial or site-to-site variation: The inherent ability of different locations or sites to support the resource of interest varies across the region of interest.  Some locations have a greater capacity to support a higher level of the target metric than others, e.g., some locations have a much greater capacity to support high abundances of particular salmon populations than others.  Describing this fundamental site to site variation (e.g, frequency distribution of abundance across sites) allows us to estimate various properties of the indicator such as its totals (total number of fish), central tendencies (mean, median) with implications with respect to precision of making these estimates.  Well designed sample surveys allow us to estimate spatial variation with known precision.  (Note that if we are interested in the properties of a single site, spatial variation is irrelevant).
• Synchronous temporal variation (across years): Some environmental driving processes affect the “level” of a metric consistently at all locations throughout the domain of interest during a particular study period.  For example, “good” ocean conditions might result in relatively high abundances in a salmon population at all sites in its domain compared with abundances following “poor” ocean conditions.  Or the thermal regime in streams during warm years can be higher across all streams in the domain compared with the thermal regime during cold years.  This synchronous temporal component of variation can significantly reduce trend detection capability. An unfortunate aspect of this component is that refining the monitoring design (e.g., increasing sample size by adding sites to the survey) does little, if anything, to reduce its effect.  Two options are available: estimate the temporal pattern in the driving process (allowing removing, or factoring out, its effect in the analytical phase), or extending the temporal monitoring interval.
• Interaction variation: We are all aware that metric values at all sites do not increase or decrease by the same amount year to year—the year to year variation among sites is variable, with metrics at some sites increasing more than at others while at other sites, metric scores decrease, some more than others.  This “independent” temporal variation among sites is captured as a sites-by-years interaction.  In general, interaction variation is much higher than synchronous temporal variation and can affect both status estimation and trend detection.  However, its effect on trend detection can be managed by the number of sites incorporated into the survey. Its effect on status estimation can be corrected with a procedure called deconvolution (Kincaid, et al., ).
• Residual variation:  We lump the remaining variation into a “residual” term that describes the variation in our ability to quantify the metric at a site during a year (during the temporal unit). Residual variation arises from our inability to perfectly describe the natural situation because our measurement methods (e.g., mark-recapture) or instruments (stream-flow gauges) have inaccuracies.  Our sampling crews do not apply field protocols in exactly the same way.   As well, we cannot sample all locations simultaneously.  If necessary, residual variation can be divided into these parts: 
• Variation in repeated applications of the measurement protocol at the same time and location by the same sampling team or introduced by the analytical methods
• Variation introduced by different crews applying the measurement protocol
• Temporal variation during the interval over which the sites are monitored within the index period.

Like interaction variation, residual variation can affect both estimates of status and trend, and similar procedures can be used to adjust for its effects. If residual variation is a relatively large part of total variation, decomposing residual variation might lead to improvements, for example, in the particular field protocol, or in greater field crew training.   Spending time improving field protocols or on team training might not be cost effective if residual variation is relatively low, or if residual variation is primarily a result of temporal variation during the index window. 

If you do not have good information on any of these four categories, do not despair.  This is the purpose of "pilot studies" (i.e. to conduct preliminary field sampling). The resulting initial estimates of these four quantities are then used to refine your design.  Consider developing your survey designs in an adaptive manner, first with some priority given to estimating four important components of variation, either as part of an initial survey, or as you can compile from existing information.  Develop sound cost estimates.  The following general framework will help get you going:  Select 30 to 50 sites using randomization in the site selection process; adopt a panel design to allocate sampling across years (e.g, some sites sampled annually, some sampled less frequently; ....click here for more information about panel designs).   After about five years, you will have a reasonable amount of information to evaluate spatial, temporal, and (interaction plus residual) variation.  Incorporating revisits to sites within a year will allow you to separate interaction from residual variation; revisiting 10 sites each year during the index window yields an effective sample size of 50 to separate residual from interaction variation.  Evaluate the potential for a “crew” effect by having different crews sample the same location at nearly the same time.  This five year set of data will allow you reasonable estimates of the components of variation that can then be used to optimize your allocation of sampling effort across sites, years, and index periods.

 

3. Monitoring Design Constraints

Constraints on monitoring designs arise from factors that might prevent you from implementing certain types of spatial, temporal, or response designs. For instance, you may lack access to potential sampling sites, be unable to collect data during certain periods, or lack resources or techniques to gather certain types of data.  By identifying these constraints now, you will have a better understanding of what type of monitoring program is feasible, and whether it will achieve your goals and objectives.  Again, these constraints apply to non-salmonid indicators that you may be monitoring, as well as salmon ones.

(J) What constraints are associated with accessing sample sites within your statistical population?

Accessibility Constraints:

(K) What constraints do you have in collecting data at different times?

Temporal Constraints:

(L) What constraints are associated with developing your metrics or collecting data for them?

Metric Constraints:

(M) What constraints do you have in funding your monitoring?

Funding Constraints:

Next:  Statements of goals & objectives | Results and next steps  | Go Back