Impact of Open Access on Citations Still Mixed

The Chronicle of Higher Education reports in its Wired Campus blog on a recent study that finds no significant citation advantage for open access articles compared to traditional subscription based articles.  This runs counter to other studies that suggest open access articles do see higher citation rates. But a mixed record of citation impacts should be no surprise to those who have followed this research area.

The headline-level synopsis of this and other studies attempting to discern the impacts of open access research obscure a multitude of methodological concerns.  They also downplay significant differences found when breaking the data down by field, country, or other factors.

In the particular study at issue, roughly one-fifth of over 3,200 articles in 36 subscription-based journals were randomly made open access over 2007 and early 2008.  (It is perhaps worth noting that the journal that published this study is produced by FASEB, an association of scientific societies, many of whom have fee-based journals that underwrite their operations.)  While the study found a notable increase in downloads, there was no notable increase in citations.

In contrast, a much more comprehensive study found an eight percent increase in citations overall, with much greater increases in citations for researchers in developing countries.  The lead researcher on the first study has speculated that most people that would be making the citations do not have issues with accessing the literature, which sort of supports the conclusions of the second study.

From a policy perspective, these results do not mean that open access is a failure.First, there is the issue of citation impacts for specific fields or locations of researchers.  It certainly seems that more research is being made available to researchers that had not been able to access it before, regardless of the overall impact.  But even if this was not the case, the policy is still not a failure; because increasing the citations of research is not the sole purpose of encouraging open access.  The increase in downloads and other indications that more people are looking at available research is important, and arguably has motivated most of the push by the U.S. government to make more of the research it funds made freely available within months of publication, if not sooner.

What’s also worth noting is this line of discussion is the reminder that citation counts are an incomplete measure of article impact.  For instance, with a much better ability to measure readership of journal articles, shouldn’t that make a difference in judging the spread of knowledge as well?  This links to a related issue I’m trying to articulate about research citations in non-academic literature that I hope to be able to post sometime this month.  Whether I’m successful with that or not, I think revisiting what is measured (and therefore valued) in research assessments is overdue.


3 thoughts on “Impact of Open Access on Citations Still Mixed


    Many studies have reported a statistically significant increase in citations for articles whose authors make them OA by self-archiving them. To show that this citation advantage is not caused by OA but just a self-selection artifact (because authors selectively self-archive their better, more citeable papers), one must first replicate the advantage itself, for the self-archived OA articles in one’s sample, and then show that that advantage is absent for the articles made OA at random. The Davis study showed only that the citation advantage was absent altogether in its sample. The most likely reason for that is that the sample was much too small (36 journals, 712 articles randomly OA, 65 self-archived OA, 2533 non-OA).

    In a recent study Gargouri et al 2010 controlled for self-selection using mandated (obligatory) OA rather than random OA. The far larger sample (1984 journals, 3055 articles mandatorily OA, 3664 self-archived OA, 20,982 non-OA) revealed a statistically significant citation advantage of about the same size for both self-selected and mandated OA.

    (There are complications with the interpretation of the Evans/Reimer study.)

    Davis, PM (2011) Open access, readership, citations: a randomized controlled trial of scientific journal publishing FASEB fj.11-183988

    Gargouri, Y., Hajjem, C., Lariviere, V., Gingras, Y., Brody, T., Carr, L. and Harnad, S. (2010) Self-Selected or Mandated, Open Access Increases Citation Impact for Higher Quality Research  PLOS ONE 5 (10)

  2. Stevan also details his methodological criticisms of the Davis study at his blog. See the entries for April 2 and March 31, 2011 for the specifics.

  3. Pingback: Narrow Measures In, Narrow Impact Out – What Gets Left Out? « Pasco Phronesis

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.