(Image credit - ARPA-E)
National Academies Report Affirms ARPA-E’s Early Progress
JUN 16, 2017
A National Academies assessment found that although it is too early to tell if ARPA–E will achieve its long-term goal of jump-starting transformative energy technologies, the agency has made clear progress with sound organizational practices.
Director of Science Policy News,
AIP
On June 13, the National Academies released a report that assesses the Department of Energy’s Advanced Research Projects Agency–Energy. The report presents findings and recommendations that aim to assist ARPA-E in continuing “to strengthen and build upon its early success.”
The committee’s assessment comes at a critical time for ARPA–E, as President Trump’s fiscal year 2018 budget proposes to terminate the $306 million agency.
Panel praises ARPA–E practices; ‘too early’ to see transformative effects
Congress originally authorized ARPA–E in the 2007 America COMPETES Act, following a recommendation by the National Research Council in the “Rising Above the Gathering Storm” report to create a new agency that would incubate innovative energy technologies. In its authorization, Congress required an independent assessment of ARPA–E after its first six years of operation. The agency received its first funding in the American Recovery and Reinvestment Act of 2009.
The National Academies assessment concludes that there are “clear indications the agency is making progress toward its statutory mission and goals.” It notes,
It cannot reasonably be expected [of ARPA–E] to have completely fulfilled those goals given so few years of operation and the size of its budget. Importantly, especially at this early stage, the committee found no signs that ARPA–E is failing, or on a path to failing, to deliver on its mission and goals.
The report observes that, since the role of ARPA–E is to fund research of potentially transformative energy technologies, it is difficult at this point to assess what, if any, transformative impacts the technologies will have in the future. Thus, the assessment chose other metrics like “organizational attributes, specific actions that are known to support and encourage innovation, and intermediate outputs” by which to assess the agency.
Using these metrics, the study committee found that ARPA–E successfully identified many “off-roadmap” projects, the majority of which would not otherwise have been funded. Additionally, ARPA–E gave its program directors the authority to take risks, initiate focused technology programs, and continuously engage with the projects they fund. Moreover, the study committee found ARPA–E’s three-year program director terms help ameliorate the tendency to narrowly focus on any one category of energy technology. Also, the agency’s “multifaceted” approach to awarding grants results in projects that contribute to ARPA–E’s goals in a variety of ways.
The report notes other parts of DOE and the federal government have adopted some of ARPA–E’s practices. Asked at the report release event whether the DOE Office of Science should consider implementing term-limited program directorships, study committee chair Pradeep Khosla responded that “every office has its culture … The leadership of the office has to decide what they want to adopt and how it best fits into their culture.”
Agency faces challenges building ‘transformational’ portfolio
The report identifies challenges that ARPA–E faces in assembling an appropriate project portfolio. At the National Academies release event for the report, study committee member Louis Schick repeated the report’s finding that ARPA–E is “holding itself to a metric of success that is not aligned with its authorizing statute.” Schick clarified there has been “enormous pressure” for each project to be transformational, which “overly narrows the activity space that ARPA–E can participate in.” He said ARPA–E should aim for a transformational portfolio rather than require every project to satisfy that criterion.
The report also recognizes that there is significant ambiguity in what constitutes a transformational project, requiring the study committee to formulate its own definition of the term. The report defines transformational projects as ones that “must be structured to seek forks in the technology roadmap [and] must challenge the conventional understanding of what is possible, practical, or profitable in impactful applications.”
The report lists several ARPA–E programs that met initial requirements for being transformational but were closed out early after being unable to reach their projected milestones. None of the agency’s projects have yet met the definition of having transformed the energy technology landscape. However, the report finds that ARPA–E projects have nevertheless produced more patents on average than projects of either DOE’s Office of Science or Office of Energy Efficiency and Renewable Energy, and more scientific publications than those of EERE while being on par with the Office of Science.
Report emphasizes communicating successes and lessons learned
While the report finds ARPA–E appears to be operating successfully, it also suggests that it “is doing a poor job of creating awareness of [its] very real successes” for general, non-technical audiences. The report stresses the importance of improving this communication since “energy issues often are at the forefront of public debates.” The report recommends engaging communication experts to help bridge this gap.
Even though some of ARPA–E’s projects have been terminated and none have yet proven transformative, Schick emphasized all projects hold significant value for the entire energy technology community, saying,
Building the communal knowledge of what has been tried and how it worked or failed is part and parcel to advancing the technology.
