Narrow Measures In, Narrow Impact Out – What Gets Left Out?

As I promised/threatened at the beginning of April, I’ve been thinking about how citations of scientific and technological research in non-academic publications aren’t captured.  Or rather, how they aren’t captured in the same was as citations in journals are.  I’m far from the first one to try and work through this, though I think there’s a lot of conventional thinking to overcome in this area.  What follows is a bit more ‘thinking out loud’ than usual, so please bear with me.  Comments are always welcome here (email – pasco dot phronesis at yahoo dot com), and if there was a post that I’d really like discussion over, this would be it.

To help with this, I think it’s useful to go back to the fundamental criteria citation metrics are trying to measure – the ‘impact’ of a piece of research.  There is an assumption that more valuable research correlates with more cited research.  There are problems with this assumption, of course, but it seems reasonable that the more a paper is cited, the more influence it has on subsequent research.  With the bias in research publications toward positive results, the presumed influence is likely positive as well.

However, this kind of measure is not focused on impact in a broad sense, but on the impact on subsequent research.  What other impacts could scientific research have, and which of those impacts would be valuable in evaluating the return on research funding?  Which of those other impacts would be valuable in evaluating researchers?  From where I am in the big research universe, it seems like these questions are in the realm of what Donald Rumsfeld once called – unknown unknowns.  We don’t know what we don’t know, and widening our measuring tools can help.

Continue reading


Stem Cell Injunction Is No More; Lawsuit Proceeds

Earlier today the U.S. Court of Appeals for the District of Columbia Circuit ruled 2-1 to overturn the injunction that would have barred federally funded stem cell research while a lawsuit over such research is tried in court (H/T ScienceInsider).  The judge in the lawsuit had issued an injunction, which the government appealed.  At the request of the government, the injunction was stayed (not implemented) pending the Court’s ruling today.  So there was very little disruption to federal funding, at least where the lawsuit is concerned.

The panel of judges that ruled today were all appointed by Republican presidents.  One of them, Judge Griffith, had heard previous arguments about the injunction in September, at a hearing where the court decided whether the stay would be implemented pending the appeal of the stay (yeah, I’m dizzy too).  The court’s ruling is based on oral arguments that took place last December (which I completely missed).  Should plaintiffs wish to press their case for an injunction, they could ask for the full Circuit Court to review the matter and/or appeal to the Supreme Court.  The split ruling from the panel may give plaintiffs sufficient motivation to try.

It should be noted that the lawsuit, which is in Federal District Court, continues.  Both sides have moved for summary judgment, but as I noted when posting about those motions last fall, it’s far from certain that such motions will be granted in this case.

As to the specifics of today’s decision, the ruling focused primarily on whether or not the injunction was warranted under relevant legal tests.  Continue reading

Just Because Government Has Science Advisers Doesn’t Mean It Will Use Them

Word from the United Kingdom (also via the BBC; H/T Xameerah Malik) that the Chief Science Adviser for the Home Office was not consulted about the announced closure of the U.K. Forensic Science Service.  The service is a wholly owned government company that provides forensic services to police departments in England and Wales.  This lack of consultation (which extends to the government’s Chief Science Adviser) came to light during a hearing of the House of Commons Select Science and Technology Committee.

As the BBC article explains, the decision was made on commercial and legal grounds, and not motivated by scientific concerns.  I have no idea if the government’s decision would have been changed had the relevant science advisers been consulted on the potential losses (whatever they are).  The U.K. government is taking pretty serious austerity steps, and if the Service is indeed projected to run out of money, I understand why the government would consider shutting it.

This is, however, not the first time government science advisers in the U.K. (or anywhere, really) have not been consulted on policy decisions where they may have useful insight.  (Both advisers were informed about the decision.)  It is not enough to have science advisers in order to demonstrate a commitment to science advice.  You have to use those advisers in meaningful ways, and not just when it’s convenient.

Canadian Science Writers Try to Get Election Attention

The President of the Canadian Science Writers Association (CSWA) has issued an open letter to the leaders of the major Canadian parties.  In the letter, Kathryn O’Hara calls for an ‘unmuzzling‘ of Canadian government scientists, that they should be made more freely available for interviews and not required to obtain clearance from public affairs offices or other non-scientist officials prior to speaking with the media.

The letter outlines some cases of this ‘muzzling’ and I am reluctant to put this in the same category of science politicisation as the editing (intentional or not) of scientific results done by various political appointees.  As I discussed over at The Bubble Chamber on a similar case, government scientists are employed under different expectations and obligations compared to their academic colleagues.  An expectation of academic freedom for scientists not working for a university is asking too much, unless it’s written into a contract or into relevant laws and regulations.

I’d be more sympathetic to the complaints in this kind of case if the research that science writers are seeking interviews about weren’t otherwise available.  As outlined in the cases cited by the Canadian Science Writers Association, the research was published, and other authors were made available for comment.  While I understand the frustration of the writers, this isn’t the same as research not being published, or edited by non-scientists prior to publication.  And it’s not unique to Canada.

The conduct of the public affairs officers mentioned here is troublesome, but it’s nothing unique to science.  (The violation of Canadian public information laws in the case discussed at The Bubble Chamber is a bigger problem, and also not unique to science.)  Think of politicians who don’t speak to the press, or only to certain media outlets.  Consider other government employees not made available for comment.

The CSWA is right to emphasize the open government angle here.  But I have my doubts that only scientists are being ‘muzzled’.  Perhaps they may have more impact in their efforts if they band with other Canadian journalism groups in a broader campaign.  As of right now, only one of the four major parties has responded to the letter.  The election is next week.

Budget Cutting Rhetoric Suggests Linear Model Still Not Dead

The American Institute of Physics has been doing its usual fine job of summarizing science-relevant appropriations and authorization busy work on the fiscal year 2012 budget in it’s FYI series (look at issues 49 and forward).  That the budget won’t be ready in time for the new fiscal year is beside the point (but worth keeping mind all the same – Congress has no credibility on the issue of timely finances and hasn’t for years).

From my observations of various committee hearings, a theme has emerged over the last couple of years within the Science, Space and Technology Committee when members are arguing requested amounts are too much.  Aside from the boilerplate about spending in this economic climate (across the board cuts are lazy and shortsighted), the theme of science agencies doing too much comes out.  It would seem like those wearing green eyeshades when looking at science funding are convinced that the government must only support basic research where its science agencies are concerned.  The report from the Budget Committee accompanying the House Republicans’ budget plan for fiscal year 2012 captures most of this sentiment pretty well:

“Included were some areas, such as biological and environmental research, that could potentially crowd out private investment. The resolution levels support preserving the Office of Science’s original role as a venue for groundbreaking scientific discoveries, while paring back applied and commercial research and development.”

Continue reading

Science and Technology Guests on Late Night, Week of April 25

The qualifying guests this week are return offenders, in that the only entrants are on repeat airings.  Tonight’s Conan rerun features animal expert Nigel Marven from early March.  Tomorrow night Dr. Sanjay Gupta’s appearance from last month on The Late, Late Show is on again.  In a rare break from current late night practice, Larry King sticks around for Dr. Gupta’s segments.

Calls for Science Service From Outside the Usual Suspects

Department of Homeland Security Secretary Janet Napolitano gave an address in March at the Massachusetts Institute of Technology.  You can read a version of that speech at Inside Higher Ed (H/T Scientists and Engineers for America).  At first glance I compared this to the op-ed by the Agriculture Secretary arguing in favor of climate change legislation – additional evidence that science and technology issues don’t respect disciplinary silos (and they do exist outside the academy).

Secretary Napolitano makes a decent argument for why science and technology matter to the Department of Homeland Security.  Her conception of better (or any) career tracks for scientists and technologists at federal agencies is worth repeating, because as the Department of State has shown, such tracks can be beneficial, but take time to develop and integrate into the rest of the agency.

However, there’s a simplistic conception of science and technology in Secretary Napolitano’s remarks that gives me pause:

“Pulling actionable intelligence from this data requires the constant evolution of our information gathering, learning, and analytic capabilities. It requires software engineers, information systems designers, and communications and data security experts working together.”

I don’t disagree with the points, but I consider them incomplete.  Capabilities to gather, analyze, and learn from information are by no means the sole domain of information and communications technologies.  Clearly the disciplinary silos are strong outside of the academy, if Secretary Napolitano thinks its only up to those disciplines.

But even if DHS is only looking for scientists and engineers in a few specialties, it certainly enforces the truth that while the National Science Foundation and the National Institutes of Health are considered the big U.S. science agencies, they don’t do all of the work.