Researchers Skeptical of NSF Metascience Plan

A proposed metascience office at the National Science Foundation has drawn concern from researchers over potential threats to peer review and political interference in grants.

The office will inform changes to the agency’s approach to funding and merit review, acting NSF Director Brian Stone said in a memo announcing several grantmaking changes at the agency.

Experts from outside the agency said the plan for a metascience office sounds good on paper, but some were unconvinced that NSF is truly interested in improving its grantmaking processes.

“I do think there are probably a lot of people at NSF that think there should be some office of evaluation and metascience,” said Mary Feeney, a professor of public affairs at Arizona State University and former program director for NSF’s Science of Science program. “But it needs to be an office that’s independent and trusted, and actually aiming to achieve high-value science instead of this sort of political agenda hidden behind the words ‘gold standard science.’”

Metascience is the study of how science is done and involves evaluating aspects of the process like reproducibility or incentives. The NSF office was first proposed in the president’s budget request for fiscal year 2027. The new memo says the office will “run rigorous experiments” across directorates and “operate with full transparency.”

Feeney expressed concern that the memo devalues panel peer review. She pointed to references to “golden tickets” — which allow an individual reviewer to advance a proposal that a larger panel may not approve — and EAGER and RAPID grants as potential mechanisms to circumvent peer review.

“EAGER and RAPID awards were designed for these really great intentions,” Feeney said. RAPID awards are reserved for situations where research must begin immediately due to urgency, such as natural disasters. EAGER awards focus on high-risk, high-reward research. “EAGER and RAPID, I think, are also seen as mechanisms to get around peer review because program officers often made the decisions on those. So, they were ways to get around the traditional peer review system,” Feeney said.

Feeney also expressed concern that placing the office under the Office of the Director would subject its work to political interference. “That makes it a political activity, not a scientific activity,” she said.

Joshua Rosenbloom, a professor of economics at Iowa State University and former director of NSF’s Science of Science and Innovation Policy program (SciSIP), said the memo’s support for metascience contrasts with the agency’s apparent aim to eliminate the Directorate of Social, Behavioral, and Economic Sciences, which funds science-of-science research. The president’s budget request for fiscal year 2027 proposes no funding for the directorate, and agency leaders said in April they were dissolving the directorate, according to Nature. Rosenbloom said that NSF has made one award for science of science and just a few in economics in fiscal year 2026.

“If they’re not supporting the community of people who are thinking about and asking these questions, then there isn’t really very much to support the in-house metascience activities,” Rosenbloom said.

Ian Banks, director of science policy at the Foundation for American Innovation, expressed enthusiasm for the new office and experimenting with new funding mechanisms. “It’s not like you’re trying to stand up a huge lab. You just need tools for data analysis and access to NSF data. So, this could be an extremely efficient way of trying to support and improve grantmaking at the agency,” he said.

Banks added that science-of-science research “is absolutely worth continuing to fund,” along with “other actually good and evidence-based” social, behavioral, and economic sciences. “I’m still a significant champion of SBE, and I’m frustrated that it was zeroed out in the president’s budget, but I still have hopes that Congress is going to appropriate for it and that we’ll see activity at SBE,” he said.

NSF did not respond to a request for comment by the time of publication.

Previous metascience efforts

Feeney said the agency has wanted — and struggled — to pursue metascience initiatives in the past because it lacked a dedicated office and budget.

NSF’s SciSIP program was located in the Office of the Director and meant to study the scientific enterprise. But eventually, the director “was using them sort of as policy experts, like you would in a legislative office, rather than this PhD-level expert in metascience,” Feeney said.

Rosenbloom said it didn’t seem like there was interest in assessing the effectiveness of some grantmaking within the agency during his time at SciSIP. “One of the challenges is that an organization has to actually want to know the answers to the questions that are being asked,” he said. When his colleagues wanted to understand the benefit of doctoral dissertation grants, “we got almost nowhere in trying to raise that question within the organization,” he said.

In 2019, NSF renamed the SciSIP program to Science of Science and moved it into the SBE directorate. Since then, the program has focused on funding metascience research rather than internal evaluation, Feeney said.

Feeney said some of the funding mechanisms proposed in the memo, such as golden tickets, have been considered before, and even informally tested by individual units.

Rosenbloom said there is already significant discussion and research on different merit review mechanisms, but agencies have been unwilling to experiment. “We don’t really ever collect data to say, how is that different? We don’t have a control group to compare it to,” he said. “I don’t see any indication that NSF is any more willing to [experiment]. They’re just simply asserting that there are better ways.”

Several interviewees pointed to the National Institutes of Health as another science agency that has been involved in metascience initiatives. Mike Lauer, former NIH deputy director for extramural research, said NIH has researched metascience topics such as mid-career investigator trajectories and grant inequalities, but has not conducted randomized trials of different funding methods.

“One of the problems, of course, is that the subjects of these experiments are real people. They’re scientists whose careers depend upon their ability to get grants. So, it’s not as simple as, say, doing an experiment on chemicals in the laboratory,” Lauer said. “When we do experiments on people in biomedical research, we have rigorous processes of informed consent, and there’s also a principle that participation in research is voluntary. Here… how would you do that?”