Developing the Report
The process for preparing condition reports involves a combination of accepted techniques for collecting and interpreting information gathered from subject matter experts. The approach varies somewhat from sanctuary to sanctuary, in order to accommodate differing styles for working with partners. The Gulf of the Farallones National Marine Sanctuary approach was closely related to the Delphi Method, a technique designed to organize group communication among a panel of geographically dispersed experts by using questionnaires, ultimately facilitating the formation of a group judgment. This method can be applied when it is necessary for decision-makers to combine the testimony of a group of experts, whether in the form of facts or informed opinion, or both, into a single useful statement.
The Delphi Method relies on repeated interactions with experts who respond to questions with a limited number of choices to arrive at the best supported answers. Feedback to the experts allows them to refine their views, gradually moving the group toward the most agreeable judgment. For condition reports, the Office of National Marine Sanctuaries uses 17 questions related to the status and trends of sanctuary resources, with accompanying descriptions and five possible choices that describe resource condition (Appendix A).
In order to address the 17 questions, sanctuary staff selected and consulted outside experts familiar with water quality, living resources, habitat, and maritime archaeological resources. A small workshop (15 participants) was convened in August 2007 where experts participated in facilitated discussions about each of the 17 questions. Experts represented various affiliations including California Department of Fish and Game and Office Spill Prevention and Response, Cordell Bank National Marine Sanctuary, Golden Gate National Recreation Area, Monterey Bay National Marine Sanctuary, National Park Service, Point Reyes National Seashore, NOAA National Marine Fisheries Service, Office of National Marine Sanctuaries West Coast Region, San Francisco Public Utilities Commission, San Francisco State University, The Institute for Fisheries Resources, U.S. Fish and Wildlife Service, Kier Associates, and the University of California Davis.
At the workshop each expert was introduced to the questions, was then asked to provide recommendations and supporting arguments, and the group supplemented the input with further discussion. When answering the set of questions, sanctuary staff and consulted experts did not consider the impacts from global climate changes. The Gulf of the Farallones sanctuary staff developed a separate site scenario document on the observed effects and predicted effects of global climate change on sanctuary resources. The site scenario served as the foundation for the site's climate change action plan which outlines strategies to reduce carbon emissions at the site, change community behavior, manage for increased ecosystem resiliency and protection, and monitor the effects of climate change. Both documents are available here.
In order to ensure consistency with Delphic methods, during the discussion a critical role of the facilitator was to minimize dominance of the discussion by a single individual or opinion (which often leads to "follow the leader" tendencies in group meetings) and to encourage the expression of honest differences of opinion. As discussions progressed, the group converged in their opinion of the rating that most accurately describes the current resource condition. After an appropriate amount of time, the facilitator asked whether the group could agree on a rating for the question, as defined by specific language linked to each rating (see Appendix A). If an agreement was reached, the result was recorded and the group moved on to consider the trend in the same manner. If agreement was not reached, the facilitator instructed sanctuary staff to consider all input and decide on a rating and trend at a future time, and to send their ratings back to workshop participants for individual comment.
The first draft of the document summarized the opinions and uncertainty expressed by the experts, who based their input on knowledge and perceptions of local conditions. Comments and citations received from the experts were included, as appropriate, in text supporting the ratings.
The first draft of the document was sent to the workshop invitees (including those who attended and those who had been invited to the workshop but could not attend) and representatives from the California Department of Public Health, California State University Monterey Bay, Gulf of the Farallones Sanctuary Advisory Council, NOAA Office of Response and Restoration, NOAA Marine Debris Program, NOAA National Marine Fisheries Service, ONMS West Coast Region, Tenera Environmental, and staff members from the Gulf of the Farallones National Marine Sanctuary for what was called an Initial Review, a four-week period that allows them to ensure that the report accurately reflected their input, identify information gaps, provide comments or suggest revisions to the ratings and text. Upon receiving those comments, the writing team revised the text and ratings as they deemed appropriate.
In September 2009 a draft final report was sent to James Allan (William Self Associates, Inc.), Rebecca Johnson (California Academy of Sciences), and John Largier (University of California, Davis, Bodega Marine Laboratory) for final review. This External Peer Review is a requirement that started in December 2004, when the White House Office of Management and Budget (OMB) issued a Final Information Quality Bulletin for Peer Review (OMB Bulletin) establishing peer review standards that would enhance the quality and credibility of the federal government's scientific information. Along with other information, these standards apply to Influential Scientific Information, which is information that can reasonably be determined to have a "clear and substantial impact on important public policies or private sector decisions." The Condition Reports are considered Influential Scientific Information. For this reason, these reports are subject to the review requirements of both the Information Quality Act and the OMB Bulletin guidelines. Therefore, following the completion of every condition report, they are reviewed by a minimum of three individuals who are considered to be experts in their field, were not involved in the development of the report, and are not ONMS employees. Comments from these peer reviews were incorporated into the final text of the report. Furthermore, OMB Bulletin guidelines require that reviewer comments, names, and affiliations be posted on the agency website. Reviewer comments, however, are not attributed to specific individuals. Comments by the External Peer Reviewers are posted at the same time as the formatted final document.
The reviewers were asked to review the technical merits of resource ratings and accompanying text, as well as to point out any omissions or factual errors. Following the External Peer Review the comments and recommendations of the reviewers were considered by sanctuary staff and incorporated, as appropriate, into a final draft document. In some cases sanctuary staff reevaluated the status and trend ratings and when appropriate, the accompanying text in the document was edited to reflect the new ratings. The final interpretation, ratings, and text in the draft condition report were the responsibility of sanctuary staff, with final approval by the sanctuary manager. To emphasize this important point, authorship of the report is attributed to the sanctuary alone. Subject experts were not authors, though their efforts and affiliations are acknowledged in the report.