Baby Steps Toward a Science of Science Policy

Two events in February demonstrated an effort to revisit the assumptions behind the processes and study of innovation. The NSF announced aProgram Solicitation in their new program on the Science of Science and Innovation Policy. Submissions are due May 22. This has been in the works at NSF since 2006, and is at least in part a response to the Office of Science and Technology Policy Director John Marburger’s call for a ‘science of science policy.’ This idea was explored previously on Prometheus. Besides the NSF program, there is a Department of Commerce advisory committee I posted about earlier that is working on how to better measure innovation.

I think both programs are good steps toward a better understanding of science policy, but are at best preliminary steps. (Any judgments about a program that has yet to receive its first grant proposals are by their nature preliminary, so please bear with me). I’ll address this in just a bit, but first some details on the two programs.

The Advisory Committee on Measuring Innovation in the 21st Century held their first meeting February 22 in Washington, D.C. The agenda, members, and other relevant documents can be found online. The group is focused on business and economic measures, as befits a Department of Commerce work. Much of the meeting was thinking out loud, working out what exactly the committee would develop. After discussion encompassing the different kinds of innovation, as well as the different ways companies measure that innovation, the group came to some preliminary points of consensus:

  • The committee will develop a group of metrics, the core of which will focus on productivity – total output per unit of total input, not the traditional output per hour measurement.
  • The committee will examine changes to the system of national accounts – the series of economic statistics gathered by several different agencies.
  • Measures will cover the different kinds of innovation: user centered, firm focused, incremental, radical, process, product, etc.

The NSF Program Solicitation for the Science of Science and Innovation Policy (SciSIP) Program anticipates granting 20-30 awards in this cycle. Located in the Directorate for Social, Behavioral and Economic Sciences, “SciSIP will underwrite fundamental research that creates new explanatory models and analytic tools designed to inform the nation‚Äôs public and private sectors about the processes through which investments in science and engineering (S&E) research are transformed into social and economic outcomes. SciSIP‚Äôs goals are to understand the contexts, structures and processes of S&E research, to evaluate reliably the tangible and intangible returns from investments in research and development (R&D), and to predict the likely returns from future R&D investments within tolerable margins of error and with attention to the full spectrum of potential consequences.”

A tall order, and this solicitation will hopefully be the first of many to really pull all of these pieces together (and support all those innovation scholars casting about for grant money). This iteration of the program focuses on Analytical Tools and Model Building, appropriate first steps for what could be an long-term exploration. There are also two special criteria for proposals: Fit to SciSIP (how the project will add to the fundamental knowledge base and Multidisciplinarity and Interdisciplinarity (encouraged but not required).

The NSF solicitation is, in my opinion, written as though they are trying to build a new body of scholarship. But such a body of knowledge and scholars is out there, and could use a solid aggregation and synthesis. But that’s not breakthrough research, and by conventional wisdom not the Foundation’s business. The NSF has also been burned (or is at the very least timid) when engaging with policy relevant research (see their workforce estimates from the early nineties). Because of those points, I am concerned that this program won’t go as far as it needs to accomplish its ultimate goals: “developing usable knowledge and theories of creative processes and their transformation into social and economic outcomes as well as developing, improving and expanding models and analytical tools that can be applied in the science policy decision making process“. (Boldface mine) Research without consideration of policy applications is one thing. But this solicitation states that policy considerations are relevant, and the NSF does not have a history of making those connections very well. How will this play out in grant applications, proposal review and awards? I’m skeptical the NSF, or the researchers applying to it, will be quick to adjust.

Both the NSF and Department of Commerce efforts are good programs that could change the way we consider innovation and policies meant to encourage it (although the implementation of the NSF program could fail to meet its intended goals – its early). But the effort to better understand investments in scientific and technological research goes beyond innovation, even beyond science and technology research. It also involves policy research, and without having that as part of the entire process, we will not have a science of science policy, but more science of innovation. It’s unclear that this is being considered. I asked Dr. Marburger what the next steps were in developing the science of science policy, and he referred me to ongoing efforts in Europe. I hope that enterprising institutions and individuals can take the work done here and grow it into a true science of science policy.