A peer review is
|“||an independent evaluation of an information product conducted by one or more technical experts.||”|
|“||[a] review of a specific component of a plan by personnel (other than the owner or author) with appropriate technical or business knowledge for accuracy and completeness]].||”|
|“||[a] [r]eview of a project or phase of a project by individuals with equivalent knowledge and background who are not currently members of the project team and have not participated in the development of the project.||”|
A peer review is
|“||[a] method of selecting manuscripts for publication in which a body of professionals or scholars is given the task of reviewing the submitted materials.||”|
Peer review is one of the important procedures used in science to ensure that the quality of published information meets the standards of the scientific community. It is a form of deliberation involving an exchange of judgments about the appropriateness of methods and the strength of the author's inferences. Peer review occurs when a draft product is reviewed for quality by specialists who were not involved in producing the draft.
The peer reviewer's report is an evaluation or critique that is used by the authors of the draft to improve the product. Peer review typically evaluates the clarity of hypotheses, the validity of the research design, the quality of the data collection procedures, the robustness of the methods employed, the appropriateness of the methods for the hypotheses being tested, the extent to which the conclusions follow from the analysis, and the strengths and limitations of the overall product.
Peer review has diverse purposes. Editors of scientific journals use reviewer comments to help determine whether a draft scientific article is of sufficient quality, importance, and interest to a field of study to justify publication. Research funding organizations often use peer review to evaluate research proposals. In addition, some federal agencies make use of peer review to obtain evaluations of draft information products that contain important scientific determinations.
Peer review should not be confused with public comment and other stakeholder processes. The selection of participants in a peer review is based on expertise, independence, and the absence of conflict of interest. Furthermore, notice-and-comment procedures for agency rulemaking do not provide an adequate substitute for peer review, as disinterested experts — especially those most knowledgeable in a field — often do not file public comments with federal agencies.
The critique provided by a peer review often suggests ways to clarify assumptions, findings, and conclusions. For instance, peer reviews can filter out biases and identify oversights, omissions, and inconsistencies. Peer review also may encourage authors to more fully acknowledge limitations and uncertainties. In some cases, reviewers might recommend major changes to the draft, such as refinement of hypotheses, reconsideration of research design, modifications of data collection or analysis methods, or alternative conclusions. However, peer review does not always lead to specific modifications in the draft product. In some cases, a draft is in excellent shape prior to being submitted for review. In others, the authors do not concur with changes suggested by one or more reviewers. Peer review may take a variety of forms, depending upon the nature and importance of the product. For example, the reviewers may represent one scientific discipline or a variety of disciplines; the number of reviewers may range from a few to more than a dozen; the names of each reviewer may be disclosed publicly or may remain anonymous (e.g., to encourage candor); the reviewers may be blinded to the authors of the report or the names of the authors may be disclosed to the reviewers; the reviewers may prepare individual reports or a panel of reviewers may be constituted to produce a collaborative report; panels may do their work electronically or they may meet together in person to discuss and prepare their evaluations; and reviewers may be compensated for their work or they may donate their time as a contribution to science or public service.
For large, complex reports, different reviewers may be assigned to different chapters or topics. Such reports may be reviewed in stages, sometimes with blinded, confidential reviews that precede a public process of panel review. As part of peer review, there may be opportunity for written and/or oral public comments on the draft product.
The results of peer review are often only one of the criteria used to make decisions about journal publication, grant funding, and information dissemination. For instance, the editors of scientific journals (rather than the peer reviewers) make final decisions about a manuscript’s appropriateness for publication based on a variety of considerations. In research-funding decisions, the reports of peer reviewers often play an important role, but the final decisions about funding are often made by accountable officials based on a variety of considerations. Similarly, when a government agency sponsors peer review of its own draft documents, the peer review reports are an important factor in information dissemination decisions, but are rarely the sole consideration. Agencies are not expected to cede their discretion with regard to dissemination or use of information to peer reviewers; accountable agency officials must make the final decisions.