1.0.1 Components of variation

We often hear that variability affects our ability to describe the metrics for which we’ve designed our monitoring programs.  However, the term "variability" connotes so much that before we can make any progress evaluating how variability affects estimation, we need to describe what we mean by “variability”.  When we consider a monitoring program whose goals include describing ecological resources that are distributed across regions across years, and for which our measurements are imperfect, organizing variability helps us focus our attention on those aspects of variation that are of specific interest, and those that impose imprecision or bias on our descriptions.  The following is one way of decomposing variability.  This brief overview is based on designing a monitoring program for status and trends of your indicators.  Consult the following for additional insights: Dauwalter, et al., 2009; Urquhart, et al., 1998; Urquhart and Kincaid, 1999; Wagner, et al., 2007; Wiley, et al., 1997.

You will need information on four broad categories of variability that will be useful for designing your monitoring program.  The information might be available from existing studies or you might design your monitoring program to obtain the necessary data during its first few years.  The following text uses "year" as the temporal unit for ease of discussion and because readers are likely familiar with programs that use "year" as the primary temporal unit. 

• Spatial or site-to-site variation: Some locations have a greater capacity to support a higher level of the target metric than others, e.g., some locations have a much greater capacity to support high abundances of particular salmon populations than others.  Describing this fundamental site to site variation (e.g, frequency distribution of abundance across sites) allows us to estimate various properties of the indicator such as its totals (total number of fish) or central tendencies (mean, median) the precision of these estimates.  Well designed sample surveys allow us to estimate spatial variation with known precision.  (Note that if we are interested in the properties of a single site, spatial variation is irrelevant).
• Synchronous temporal variation (across years): Some environmental processes have a consistent impact at all locations throughout the domain of interest during a particular study period.  For example, “good” ocean conditions might result in relatively high abundances in a salmon population at all sites in its domain compared with abundances following “poor” ocean conditions.  Or the thermal regime in streams during warm years can be higher across all streams in the domain compared with the thermal regime during cold years.  This synchronous temporal component of variation can significantly reduce trend detection capability because it adds more noise (i.e. variability) to your data.  An unfortunate aspect of this component is that refining the monitoring design (e.g., increasing sample size by adding sites to the survey) does little, if anything, to reduce its effect.  Two options are available: 1) estimate the temporal pattern in the driving process that will allow you to remove, or factor out its effect in the analytical phase; or 2) extend your monitoring program because the longer you monitor the more sensitive your analysis will be for detecting trends.
• Interaction variation: We are all aware that metric values at all sites do not increase or decrease by the same amount year to year—the year to year variation among sites is variable, with metrics at some sites increasing more than at others while at other sites, metric scores decrease, some more than others.  This “independent” temporal variation among sites is captured as a sites-by-years interaction.  In general, interaction variation is much higher than synchronous temporal variation and can affect both status estimation and trend detection.  However, its effect on trend detection can be managed by the number of sites incorporated into the survey. Its effect on status estimation can be corrected with a procedure called deconvolution (Kincaid et al. 2004).
• Residual variation:  We lump the remaining variation into a “residual” term that describes the variation in our ability to quantify the metric at a site during a year (during the temporal unit). Residual variation arises from our inability to perfectly describe the natural situation because our measurement methods (e.g., mark-recapture) or instruments (stream-flow gauges) have inaccuracies.  Our sampling crews do not apply field protocols in exactly the same way.   As well, we cannot sample all locations simultaneously.  If necessary, residual variation can be divided into these parts: 
• Variation in repeated applications of the measurement protocol at the same time and location by the same sampling team or introduced by the analytical methods
• Variation introduced by different crews applying the measurement protocol
• Temporal variation during the interval over which the sites are monitored within the index period.

Like interaction variation, residual variation can affect both estimates of status and trend, and similar procedures can be used to adjust for its effects. If residual variation is a relatively large part of total variation, decomposing residual variation might lead to improvements, for example, in the particular field protocol, or in greater field crew training.   Spending time improving field protocols or on team training might not be cost effective if residual variation is relatively low, or if residual variation is primarily a result of temporal variation during the index window. 

If you do not have good information on any of these four categories, do not despair.  This is the purpose of "pilot studies" (i.e. to conduct preliminary field sampling). The resulting initial estimates of these four quantities are then used to refine your design.  Consider developing your survey designs in an adaptive manner, first with some priority given to estimating four important components of variation, either as part of an initial survey, or as you can compile from existing information.  Develop sound cost estimates.  The following general framework will help get you going:  Select 30 to 50 sites using randomization in the site selection process; adopt a panel design to allocate sampling across years (e.g, some sites sampled annually, some sampled less frequently; ....click here for more information about panel designs).   After about five years, you will have a reasonable amount of information to evaluate spatial, temporal, and (interaction plus residual) variation.  Incorporating revisits to sites within a year will allow you to separate interaction from residual variation; revisiting 10 sites each year during the index window yields an effective sample size of 50 to separate residual from interaction variation.  Evaluate the potential for a “crew” effect by having different crews sample the same location at nearly the same time.  This five year set of data will allow you reasonable estimates of the components of variation that can then be used to optimize your allocation of sampling effort across sites, years, and index periods.

Go Back