House Subcommittee Considers International Science Benchmarking
How much money is enough for U.S. science, and how should policymakers determine what level of investment is appropriate for each field? Even as Congress searches for an exit strategy to finish its work for the year and head home to campaign, the House Science Subcommittee on Basic Research considered ways to address these perennial questions at an October 4 hearing on international benchmarking of science.
A committee of the National Academies earlier this year completed a study exploring the feasibility of using international benchmarking - assessing the country’s relative strength in a given research field versus other countries - as an input to such policy decisions. The Committee on Science, Engineering, and Public Policy (COSEPUP) set up groups of experts to evaluate the U.S. position in three fields: mathematics, immunology, and materials science and engineering. This study, published in March, was prompted by a previous (1993) COSEPUP report that recommended research investment decisions be made on the basis of keeping the U.S. “among the world leaders in all major areas of science,” so it is positioned to take rapid advantage of breakthroughs wherever they occur, and selecting certain strategic fields in which the U.S. should be the clear world leader.
Easily-gathered quantitative data, such as number of citations, publications, researchers and graduate students supported, and level of funding, is not adequate to determine the nation’s position relative to other countries, testified Mary Anne Fox, chair of the COSEPUP panel that organized the benchmarking studies. Comparable data is often not available for other countries, and quantitative indicators alone do not give an accurate picture of the health of a field, she said; subjective analysis by knowledgeable experts is also needed. Using committees of such experts, the COSEPUP panel concluded that performing benchmarking is a “rapid and inexpensive” way of assessing the relative U.S. position in a field. The case studies showed the nation to be among the world leaders in all three selected fields, reported COSEPUP member Robert White of Carnegie Mellon. The studies, however, also identified some areas of potential concern: leadership in math is dependent on foreign researchers who came to the U.S. because of World War II and the collapse of the Soviet Union. In immunology, the U.S.'s managed care systems make it difficult to attract patients needed to conduct clinical trials. In the field of materials, many U.S. facilities are 20-30 years older than comparable facilities in Europe and Japan.
While commending COSEPUP’s efforts, National Science Board (NSB) Chairman Eamon Kelly remarked that “international benchmarking, while an important component...is not sufficient by itself to tell us how much the government should invest in a given field.” In addition to international position, he said, policymakers should take into account the potential public benefits, the health of the infrastructure, the opportunities for rapid advancement, and the government’s need to perform specific missions, in determining how much money a discipline should receive. He added that the NSB is currently conducting a study to examine just such factors and how they can aid the decision- making process.
Even though this was the subcommittee’s last hearing of the 106th Congress, Chairman Nick Smith (R-MI), Ranking Minority Member Eddie Bernice Johnson (D-TX), Technology Subcommittee Chairwoman Connie Morella (R-MD) and others on the subcommittee clearly share an abiding concern over how best to make decisions affecting the federal research portfolio. Smith indicated an interest in possibly having COSEPUP perform benchmarking studies for the government on a permanent basis. As he thanked Johnson for her service on the subcommittee, and she said she looked forward to returning next year, Smith added a joking reference to the upcoming elections: “We’ll see who is chair four months from now.”
The March 2000 COSEPUP study, “Experiments in International Benchmarking of U.S. Research Fields,” runs 316 pages and is available for viewing or ordering online at http://books.nap.edu/catalog/9784.html
