The American Institute of Physics Bulletin of Science Policy NSTC Report Addresses the Challenge of "Assessing Fundamental Science"
FYI
Number 140: October 2, 1996
As federal budgets grow ever tighter, the themes of accountability and
efficiency with the taxpayers' money are receiving increased attention. By
fiscal year 1999, all federal departments and agencies will be required to
produce a report assessing progress toward their stated goals, as called for
in the Government Performance and Results Act (GPRA) of 1993 (see FYI #106.)
While some federal programs are amenable to judgment by quantitative measures,
it is widely recognized that the performance of the basic research enterprise
cannot be evaluated so easily. Outcomes and impacts of basic science are
frequently not apparent for decades, and can come in unexpected ways and
unpredictable areas. With the help of the Office of Science and Technology
Policy (OSTP) and the National Science and Technology Council (NSTC), the
government agencies that perform science have implemented pilot programs to
develop metrics and methods for judging the government's performance of
fundamental research. The data accumulated so far are described in a July
1996 report by the NSTC's Committee on Fundamental Science, entitled
"Assessing Fundamental Science."
According to the report, a central issue is defining a goal against which
progress can be measured. Because basic research contributes to myriad
national goals in a complex way and over long time periods, the Administration
has identified an intermediate, or enabling, goal against which to assess
progress: U.S. leadership across the frontiers of scientific knowledge. If
the U.S. can remain at the forefront in all areas of science, it will be
well-positioned to achieve other national goals such as improved health,
environmental preservation, economic prosperity, national security, and
quality of life. This objective, the report says, will provide "the principle
yardstick for GPRA assessment strategies for fundamental science programs."
The report admits that assessment techniques for fundamental science are still
in their infancy, but concludes from experiences so far that "merit review
based on peer evaluation will continue to be the primary vehicle" for
assessing excellence. It cautions that pre-existing quantitative metrics are
not sufficient to measure such aspects of basic science as innovation, and
warns against indicators that might discourage risk-taking and creativity.
The report sets out the following principles for evaluation: clearly-defined
program goals; criteria that will encourage excellence and responsiveness;
performance indicators that are useful and appropriate; avoidance of
burdensome and counterproductive assessments; incorporation of merit review
and peer evaluation; multiple sources and types of evidence; experimentation
to develop effective assessment tools; reports that will inform and refine
policy development; and results that can be understood by policymakers and the
public.
"The passage of GPRA offers scientists and science managers the opportunity to
adapt the best planning and management methods to build world-class science
programs," the report concludes. The Appendices discuss progress in pilot
programs underway in NSF, DOE, NIST, NIH and USDA. The report, which runs 71
pages including appendices, can be found on Internet at:
www.nsf.gov/sbe/srs/ostp/assess/start.htm
###############
Audrey T. Leath
Public Information Division
American Institute of Physics
fyi@aip.org
(301) 209-3094
##END##########
[an error occurred while processing this directive]
|
