House Committee Proposes Refinement of Broader Impact Criterion

Tomorrow the House Science and Technology Committee will markup an authorization bill for the National Science Foundation (NSF).  An authorization bill will set spending amounts for a specific time (fiscal years 2011 through 2015), but the appropriations committees have to actually allocate the money.

Besides the spending levels, the bill does have some specific guidance for spending on research infrastructure; science, technology, engineering and mathematics (STEM) education; and other specific research foci.  Of note is the language in section 104 on the broader impacts review criterion for grant applications.  Most of the section describes how NSF will develop and implement policy for ‘criteria 2′ that would train relevant parties on the broader impacts criterion and demonstrate more support for that criterion from both the agency and the institutions that support grant applicants.

There is a statement of goals for the Broader Impacts Criterion that caught my eye.

“(a) GOALS.—The Foundation shall apply a Broader Impacts Review Criterion to achieve the following goals:

(1) Increased economic competitiveness of the United States.
(2) Development of a globally competitive STEM workforce.
(3) Increased participation of women and underrepresented minorities in STEM.
(4) Increased partnerships between academia and industry.
(5) Improved K-12 STEM education and teacher development.
(6) Improved undergraduate STEM education.
(7) Increased public scientific literacy.
(8) Increased national security.”

While this is a more detailed description of the current broader impacts criterion (which I list below), I don’t think it really changes the criteria so much as add emphases, particularly to what is currently the benefits to society part.

Criterion 2: What are the broader impacts of the proposed activity?

• How well does the activity advance discovery and understanding while promoting teaching, training and learning?
• How well does the proposed activity broaden the participation of underrepresented groups (e.g., gender, ethnicity, disability, geographic, etc.)?
• To what extent will it enhance the infrastructure for research and education, such as facilities, instrumentation, networks and partnerships?
• Will the results be disseminated broadly to enhance scientific and technological understanding?
• What may be the benefits of the proposed activity to society?

Some important things to remember about this proposal.

It’s a proposal. It could be changed or dropped in tomorrow’s markup hearing. The hearing should be webcast, check the committee’s website tomorrow morning (hearing scheduled for 10 a.m. Eastern) to access it.

The language is about goals.  The section I quoted is not what the text of the broader impacts criterion will be, but what that criterion should achieve.  It will be up to NSF to determine how the language of the criterion and the associated guidance should change (if at all) in order to accomplish the desired goals.

The devil is in the enforcement details.  How this criterion is implemented will still matter much more than the content within.  The goals listed in this bill have the same room for interpretation as the current criterion.  The authors recognize this with all the language addressing implementation, but given the massively decentralized operation that is merit review, successfully implementing changes is the hard task here.

I suspect all the British researchers that complained about the impact calculations in their research assessments would find this proposal inappropriate.  Some in the U.S. may object as well, but I would be surprised if those that do didn’t already have a problem with the broader impact criterion.