Bad Science – Intention Matters, but is Easy to Gloss Over

(This post is still a bit rough, certainly compared to my usual work.  I’m not completely satisfied with this.  But I’m at a point where I need to get this out there and get some response, positive and/or negative)

This graphic on “the psychology of exaggerated and false research” circulated through the blogosphere a couple of weeks ago.  (Disclaimer, I was approached about the graphic by someone at ClinicalPsychology.net who developed it before it went public.)  When I first saw it my gut reaction was anger.  Anger at the gap between how some (me included) would like scientific research to be conducted and how it is.  So I Tweeted my anger.

Now there are other reasons for anger over this.  Here are a couple. You may think the graphic oversimplifies matters (always a risk).  You may think the cited facts don’t really explain things, or explain them clearly, which is what Jess at Bioephemera suggested to me.  And she’s got an excellent point.  What do they mean by “questionable methods,” “cutting corners,” or other amorphous terms?

Well, you can check the cited sources.  Unfortunately, they aren’t linked to particular parts of the graphic.  The folks at ClinicalPsychology.net helped some with that, and I’m grateful.

What is still hard to distinguish in this graphic are faulty research results that come from intentional acts from faulty results that come from researcher error.  A statistical error because someone miscalculated is a different kind of problem than someone who misrepresented data through bad statistics.  It can be difficult to distinguish these behaviors in the field/at the bench, but it is important to note and emphasize the difference between errors and abuses, between fraud and poor technique.  They are all problems, but they do not need the same solutions, and do not deserve the same outrage (which Jess helped me to remember).

Let me be clear, much of the data here still makes me mad.  But as much as anger can be an effective motivator, it’s important to be clear about what the problems are, and the different responses needed.

From here I want to talk about the related post in which I found this graphic, and the recommendations made in this graphic.  Those will be separate posts in the next few days.

About these ads

4 thoughts on “Bad Science – Intention Matters, but is Easy to Gloss Over

  1. Good points. I mentioned some of these things to the Clinical Psychology people in my response to them before posting as well, noting the irony in an infographic warning against the sloppy authoritative use of numbers, by itself presenting authoritative numbers without clear references.

    Anyway, look forward to your follow-ups.

  2. Pingback: Sure, Woo is Awful, But Does It Deserve More Attention Than Bad Science? « Pasco Phronesis

  3. Pingback: Could Anonymity Help *Fight* Bad Science? « Pasco Phronesis

  4. Pingback: There’s Good And Bad In The Rise In Retractions « Pasco Phronesis

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s