TWS Issue Statement: Scientific Peer Review of Agency Decision Processes

Back to Position Statements page

Peer review is an integral component of scientific research and publishing and an important means of assuring sound information. It allows the scientific community to maintain quality control of research through the review of research proposals, journal manuscripts, synthesis or summary reports and other documents. Academic peer review is the best tool scientists have to ensure high standards for their professional work.

This idea has been translated into the policy arena through ‘scientific peer review’ – the review, by scientific experts, of in-house agency science, synthesis reports, or the body of science underlying management decisions. These types of reviews are critically important tools for policy makers. They allow experts from both inside and outside the state and federal government to provide technical advice and analysis, increasing public confidence in government science, and ensuring that the best-quality information is used in decision-making. The role of science in policy and decision-making is to inform the decision process, rather than to prescribe a particular outcome. What is needed is an objective evaluation of the credibility and applicability of the science used to influence the policy decision.

However, it is critical that scientific peer review programs be carefully designed to maintain objectivity, quality, and thoroughness. While unbiased and rigorous scientific peer review is an important tool for decision-makers, a poorly designed process can do more harm than good. It is for this reason that we endorse the following list of important considerations for government scientific peer review of agency-produced science and the body of science underlying management decisions.

The policy of The Wildlife Society regarding peer review is that:

1. The first priority in choosing reviewers should be to engage the most competent scientists. The key issue in selecting reviewers is whether they bring the necessary scientific knowledge and objectivity to reviewing the matter at hand. Care should also be taken to ensure that, when possible, reviewers represent human diversity, as well as a diversity of views and professional experiences.
2. Conflict of interest requirements are objective standards designed to eliminate certain specific, potentially compromising situations from arising, and thereby to protect the reviewer, the other members of the review team, the agency, and the public interest Conflict-of-interest exclusions must be carefully designed to balance barring those with a direct conflict of interest and the reality of a finite pool of suitable reviewers.
3. Scientific peer review should be free of political interference and influence. Oversight of scientific peer review should be vested in scientists and managers within the agencies making decisions based on the science. There must be a competent moderator that can move forward the scientific debate of the merit of the evidence in question to achieve a reasoned decision without unduly influencing the decision. This adds assurance that the composition of panels is not being unduly influenced by politics and constitutes a representative subset of the scientists most competent to review and assess the topic. The agencies must be trusted to perform the task of constituting and overseeing fair and independent scientific peer review efforts, without interference from political entities. Agencies must not violate this trust and should realize that failure to insulate peer review from political considerations will destroy the credibility and value of current and subsequent peer reviews.
4. Even the best scientific peer review cannot give policy makers the ‘right’ answer. Scientific peer review can provide assurances that rigorous, transparent, and respected methods were followed, that the data were reasonably interpreted, and that the stated conclusions logically follow from the results. However, often more than one interpretation of the data set can be made, and there may be no way to determine which interpretation is ‘best’. Where data are limited or other uncertainties abound, scientific peer review can point these problems out, but it cannot overcome them. Scientific peer review is focused on the scientific merit of manuscripts, reports, or other documents and avoids advocacy of policy implications indicated by data and author conclusions. Science can inform but cannot supplant human value judgments when making management decisions; this distinction must remain clear. Often peer review is useful in recognizing the degree of uncertainty upon which management decisions are based. Agencies should employ adaptive management to address such uncertainty, especially where it is large and/or significant, and public resources are to be committed.
5. Scientific peer review must maintain programmatic flexibility. Guidelines can help to ensure that certain standards are met and maintained, but an overly rigid process, particularly for peer review of the body of science underlying policy decisions, will result in inefficient use of time and resources. It may be overly prescriptive to stipulate the number of reviewers, the questions they must answer, or the type of report they must produce for the broad range of agency science in research and management. Nonetheless, program or policy evaluation metrics must be objective and clearly defined so the standard of comparison is explicit.
6. All scientific peer review must be based upon an assumption of integrity. While commonsense measures can be taken to weed out direct conflicts of interest, potential conflicts of interest may be unavoidable. Fair reviews are the product of professional standards of conduct that are a fundamental component of training in scientific research and experimental management. Scientific peer review must ultimately rest on the presumed integrity of the reviewers. Certification programs are a useful process for promoting scientific integrity and ethical behavior, and the receipt of certification by an individual can provide some assurance of integrity and professionalism in a reviewer. In addition, other measures such as making majority and minority opinions or a synopsis of them, available upon request, or instituting an appeals process may be necessary.
7. Efforts to revise the process of peer review should acknowledge the differences in professional culture that often divide scientists, policy makers, and the public. The academic model of peer review calls on reviewers to be as critical as possible. This is done so that authors are able to make improvements where they can and so that the weaknesses of the work are understood and acknowledged. Thus, results from scientific peer review that highlight uncertainties, questions and alternative explanations do not mean that the science was not well done or that its findings are invalid. Science deals with reducing, not eliminating, uncertainty, and there will always be unanswered questions and areas where more research is needed. However, acknowledging uncertainty should not be equated with an inability to draw conclusions; managers often must act without complete certainty. Scientific peer review, properly carried out by competent peer scientists, can reassure managers, decision makers, policy makers and the public that such difficult decisions are based on research that represents the current state of our scientific understanding.
8. Sound science sometimes requires the use of various types of scientific data. During peer review, a variety of data types should be accepted as a basis for decisions. Sound scientific and policy decisions often require the use of diverse types of information, but should always be based on the best available data. Scientific peer review should permit the use of various data types, rather than requiring one specific type or setting up a hierarchy of acceptable data.

The Wildlife Society’s Position Statement on The Use of Science in Policy and Management Decisions supports the “establishment of laws and guidelines that ensure the appropriate use of science by policy and decision-makers.”

Scientific Peer Review of Agency Decision Processes Issue Statement pdf