NSF Will Implement New Merit Review Criteria
Last year the National Science Foundation and the National Science Board formed a Task Force to suggest changes to NSF’s merit review criteria, which had not been revised since 1981. The Task Force unveiled its proposed criteria in November 1996 (see FYIs #162 and #163 , 1996) and made them available to the scientific and engineering communities for public comment. Based on the 325 responses received over a two-month period, the Task Force revised its draft criteria and presented them to the NSB at its March 27-28, 1997 meeting. The Board approved the new criteria and authorized NSF Director Neal Lane to “proceed expeditiously with all steps necessary” to implement them for all proposals reviewed beginning October 1, 1997.
The Task Force reduced the number of criteria reviewers must consider from four to two. Each of the criteria has a set of related questions to help reviewers evaluate the proposals. The instructions make clear that the two criteria “need not be weighted equally.” Reviewers are asked to provide separate comments for each criterion, a single composite rating of the proposal, and a summary recommendation that addresses both criteria.
Based on the public comments, the Task Force altered its proposed criteria and associated questions to place more emphasis on researcher competence, and to clarify wording on issues of diversity, creativity, benefits to society, and management of the research plan. Some questions were rephrased to encourage reviewers to provide explanations rather than yes-no answers. The Task Force believes that “adoption of the new criteria will facilitate, clarify and simplify the proposal evaluation process.” The revised criteria, as approved by the National Science Board, are quoted below:
* * * * * 1. What is the intellectual merit of the proposed activity?
The following are suggested questions to consider in assessing how well the proposal meets this criterion: How important is the proposed activity to advancing knowledge and understanding within its own field and across different fields? How well qualified is the proposer (individual or team) to conduct the project? (If appropriate, please comment on the quality of prior work.) To what extent does the proposed activity suggest and explore creative and original concepts? How well conceived and organized is the proposed activity? Is there sufficient access to resources?
2. What are the broader impacts of the proposed activity?
The following are suggested questions to consider in assessing how well the proposal meets this criterion: How well does the activity advance discovery and understanding while promoting teaching, training, and learning? How well does the proposed activity broaden the participation of underrepresented groups (e.g., gender, ethnicity, geographic, etc.)? To what extent will it enhance the infrastructure for research and education, such as facilities, instrumentation, networks, and partnerships? Will the results be disseminated broadly to enhance scientific and technological understanding? What may be the benefits of the proposed activity to society?
