Controlling the Literature Review for Bias

Great piece from Ben Goldacre of Bad Science on why research from sources you don’t trust isn’t necessarily untrustworthy.  While in some cases, often those that conflict of interest policies could help manage, the initial distrust is justified.  However, when considering sources that appear untrustworthy for other reasons, it’s not so clear that their research is automatically bad.

What to do, then?  Simple – do the legwork.  As Goldacre describes in his piece, identify the background of the researchers, and examine if there are correlations between suspect researchers and their research.  Making the snap judgment without the legwork raises the possibility that valuable knowledge is buried because of who shaped it.  While “fruit of the poisoned tree” is a valid legal principle for tossing all the fruit, researchers need to examine each and every piece to see if it’s poisoned or not.

Now this can lead to difficulties when the politically legitimate move of downplaying someone by association collides with an output of properly done science.  In the case of Carlin and the EPA, the agency ended up dismissing Carlin’s input (via reprimand) without conducting the due diligence first to demonstrate that his input was not properly done.  The EPA staff was correct to dismiss the work, but jumped the gun.  The result is a formal reprimand for the relevant official, and the political exposure of the agency for those who are eager to score the political hit without regard for the validity of the work that was at issue.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s