Peterman et al. 2005

Peterman, R. M., B. Dorner, and M. A. Nelitz. 2005. Review of experimental design principles for projects to restore salmon populations in the Arctic-Yukon-Kuskokwim region of Alaska. Arctic-Yukon-Kuskokwim Sustainable Salmon Initiative, 2004-2005 Project # 45237, Anchorage, Alaska.

This document was written to support the development of a successful restoration program for low-abundance Pacific salmon stocks (Oncorhynchus spp.) in the Arctic- Yukon-Kuskokwim (AYK) region of Alaska. By "restoration program" we mean a broad set of human actions, such as new harvest regulations or improvements to salmon habitats, that aim to restore salmon populations to higher abundances or to maintain resilience to future disturbances such as climatic changes. Based on the wide experience with attempts to rebuild salmon stocks throughout the North Pacific Rim over the last 30 years, it is likely that only some of these current and future actions will succeed at their restoration objectives. The reason for doubt is that such actions are vulnerable to unexpected outcomes that result from uncontrollable natural events, as well as our incomplete knowledge of how complex salmon ecosystems function. Actions such as changing mesh size or other harvest regulations, supplementation of juvenile abundance through additions of juvenile fish from incubation boxes or hatcheries, or removal of beaver dams, may have unintended negative effects on target salmon populations through selection for earlier age at maturity, spread of diseases, or loss of deep over-wintering pools for juvenile coho salmon, respectively. Given this uncertainty about outcomes, we should design restoration projects so that their effectiveness can be rigorously evaluated in the future. For example, 5 or 10 years from now we want to be able to convincingly conclude that restoration methods 1, 4, and 5 (whatever those might be) met the restoration objectives, but methods 2 and 3 did not. Future funds could then be focused on methods 1, 4, and 5, unless methods 2 or 3 had offsetting benefits such as fostering of stewardship through local involvement, or improving other ecosystem benefits such as water quality or habitat for other species. A huge challenge for evaluation of restoration actions is created by the presence of natural variability in ecological systems and simultaneously changing (and potentially confounding) factors. These factors may obscure the actual effects of restoration on salmon populations. To reduce the chance of such confounding, a well-designed action plan could compare, for example, salmon populations from several sites to which a particular type of "treatment" had been applied (e.g. incubation boxes to increase the number of fry produced per egg) with other "control" populations that did not receive that
"treatment". Without such comparison groups, incorrect conclusions could easily be drawn about the effectiveness of some restoration action. For example, an increase in abundance of adult salmon recruits that was actually due to improved ocean conditions might be incorrectly attributed to the incubation boxes, or conversely, a lack of increase in abundance caused by deteriorating marine survival rate might lead to an incorrect conclusion that the boxes were not working, when in fact they were successful at increasing freshwater survival. Restoration actions should thus be viewed as experiments, where the intent is to
determine, through manipulation of the ecological system and follow-up monitoring, which approaches work best in terms of achieving a stated management objective. Note that this recommendation applies equally to changes in fishing regulations, to habitatoriented actions, and to socio-economic programs designed to aid restoration. Meeting the goals of thorough evaluation and accountability requires that restoration actions be planned ahead of time to conform with rigorous experimental designs, which will permit evaluation of their relative effectiveness at some later date. Without such rigorous design, it will be difficult, if not impossible, to attribute some observed response in fish populations to a particular restoration action. We review past experience and literature on the concepts of experimental design to develop a framework that will permit application of these concepts to restoration actions for salmon in the AYK region of Alaska. This document was written with two audiences in mind, those who will be undertaking restoration projects and those who will choose which projects to fund. This dual audience is consistent with an important theme of the AYK Sustainable Salmon Initiative (AYK SSI), which is to create a research and
restoration program "... that is inclusive through a strong communication plan and public involvement ..." and "... is strongly committed to seeking a more inclusive process through capacity building" (AYK SSI 2004). Thus, to address the first audience, this
paper provides a framework of concepts and components of experimental design that can be used by proponents of restoration projects (e.g. local or regional community groups, non-profit native corporations, other non-governmental organizations, and government agencies). By following this framework, restoration practitioners will be able to assess whether their project was able to meet its restoration objectives. The second audience, the AYK Scientific and Technical Committee (STC), will be writing a "Research and Restoration Plan" in the future. If experimental design criteria such as those described in this report are incorporated into that plan, it will result in the following benefits:

• The AYK STC or other funding groups will be able to evaluate proposed restoration projects based on whether they meet the requirements for good experimental design. As has been done in other restoration programs in western North America, that criterion will be an important component of funding decisions to permit evaluation of a restoration project's effectiveness.
• An implicit goal of restoration programs will more likely be met: learning more about the processes affecting AYK salmon populations.
• Accountability will increase at all levels and allow a thorough evaluation of the SSI restoration plans as a whole.
• The Arctic-Yukon-Kuskokwim Sustainable Salmon Initiative will be able to respond dynamically as understanding of regional ecosystems increases. Thus, the AYK SSI will be able to benefit from a more efficient use of its funds by focusing over the long
term on the most effective restoration methods. We recognize that in the last two years, some of the salmon populations that were a serious concern in the AYK region have recovered remarkably quickly, even without major interventions. This outcome is, of course, excellent for the salmon and people that depend on them, but it also clearly illustrates the need to conduct restoration projects (including changes in fishing regulations and other similar management actions) using experimental designs, as we describe in this document. For instance, imagine that millions of dollars had already been spent by the AYK SSI on various restoration efforts prior to 2001. If the same natural events had occurred as we have observed since then and if no provisions had been in place to monitor in an experimentally designed manner to account for the confounding effects of these events, many people would have falsely attributed the quick turnaround in salmon abundance to the restoration actions, creating unwarranted confidence in the ability of these actions to prevent or ameliorate future declines. Hence, an experimental approach to design of restoration projects is critical. Despite the rapid recovery of several AYK salmon populations, the material that we present here is still relevant because the STC's future "Research and Restoration Plan" will be developing a framework that can be used not only for those AYK salmon populations that have not yet recovered, but also for any AYK populations that may be in need of restoration in the future. In addition, our material is also relevant in the context of precautionary actions, such as changes in management practices, that may be taken to increase the resilience of AYK populations and reduce the chance of future collapses.
In this report, we use the term "restoration (or recovery) actions, activities, plans, or projects" in a broad sense to reflect a wide range of human actions. These restoration actions, which could also be considered management "levers", potentially include
changes in fisheries regulations, new management practices, alteration of freshwater habitats, various forms of enhancement, and socio-economic initiatives. These actions are intended to meet the objectives of:
• Restoring, or achieving recovery of, salmon populations to some higher level
of abundance and productivity, or
• Maintaining salmon populations' resilience by removing stressors that might
make the populations more vulnerable to future unfavorable conditions.