Controversial Topics, Metrics and Analytics, Research

Citation Controversy

Louis XIV visiting the  Académie des sciences ...

Image via Wikipedia

Earlier this year, I reported on a study by sociologist James Evans suggesting that online access to scientific journals is leading to more recent citations and a narrowing of the diversity of those articles which are cited.

This study was not taken at face value, and three information scientists (Vincent Larivière, Yves Gingras, and Éric Archambault) all at the University of Quebec in Montreal have released a new analysis taking aim at the diversity claim.

Their manuscript, “The decline in the concentration of citations, 1900-2007,” deposited September 30th in the arXiv, uses a simpler methodology. They report the percentage of papers that received at least one citation, the percentage of papers needed to account for 20%, 50%, and 80% of total citations, and the Herfindahl-Hirschman index, a measure used to estimate market concentration.  The authors divide up their articles into four broad categories (natural sciences and engineering, medical fields, social sciences, and the humanities), and report their data in 2-year, 5-year, and 10-year citation windows.  The figures are pretty unambiguous, with the exception of the humanities — all measures of diversity appear to be increasing, not decreasing.

Although the distributions of citations received are still highly concentrated and a minority of papers still account for a majority of the citations, this level of concentration has been decreasing over time

Unlike Evan’s article, this paper does not require knowledge of negative binomial regression, or any advanced statistics for that matter; and because of the simplicity and descriptive approach to their analysis, it is very convincing.  Granted, Evans is using a different approach, looking at the effect of when journals became available online on citation behavior and whether commercial access or free access changes the outcome.  For that reason, we should not discount the merit of Evan’s paper.

What makes this controversy interesting is that both studies make theoretical sense.  A narrowing of science conforms to attention economics and preferential attachment (why the cited get more citations and the rest get ignored); a broadening of science conforms to information foraging theory, the principle of least effort, and the increasing ease of retrieving relevant articles.  The results of both studies imply something different about the state of science, whether scientific information is being disseminated efficiently, and whether the literature is reflecting more diversity of opinion or more conformity.  Larivière et al. write:

[O]ne can therefore argue that the scientific system is increasingly efficient at using published knowledge. Moreover, what our data shows is not a tendency towards an increasingly exclusive and elitist scientific system, but rather one that is increasingly democratic

Like many scientific controversies, the argument over citation diversity will move toward consensus and closure.  For the meantime, the debate remains open.

Postscript (January 6, 2009)

In the January 2nd issue of the journal Science, Larivière et al. published a letter entitled “Literature Citations in the Internet Era,” in which they detail their own findings.  They write:

Although many factors affect citation practices, two things are clear: Researchers are increasingly relying on older science, and citations are increasingly dispersed across a larger proportion of papers and journals.

Postscript (April 3, 2009)

The article, “The decline in the concentration of citations, 1900-2007″ appears in the April issue of the Journal of the American Society for Information Science & Technology (2009, vol 60, p.858-862)

Reblog this post [with Zemanta]

About Phil Davis

I am an independent researcher and publishing consultant specializing in the statistical analysis of readership and citation data. I am a former postdoctoral researcher in science communication and former science librarian. http://phil-davis.org/

Side Dishes by Stewart Wills

Find Posts by Category

Find Posts by Date

October 2008
S M T W T F S
« Sep   Nov »
 1234
567891011
12131415161718
19202122232425
262728293031  

The Scholarly Kitchen on Twitter

SSP_LOGO
The mission of the Society for Scholarly Publishing (SSP) is "[t]o advance scholarly publishing and communication, and the professional development of its members through education, collaboration, and networking." SSP established The Scholarly Kitchen blog in February 2008 to keep SSP members and interested parties aware of new developments in publishing.
......................................
The Scholarly Kitchen is a moderated and independent blog. Opinions on The Scholarly Kitchen are those of the authors. They are not necessarily those held by the Society for Scholarly Publishing nor by their respective employers.
Follow

Get every new post delivered to your Inbox.

Join 14,527 other followers

%d bloggers like this: