Metrics and Analytics, Peer Review, Research, Social Role

Peer Review Survey 2009

Trusting Blind Peers
Image by Gideon Burton via Flickr

Peer review is slow, inefficient, and prone to bias. It doesn’t always detect important work, is unable to identify errors and intentional fraud, and can be abused by editors and reviewers. Peer review is an expensive and imperfect process although it remains a highly desirable component of formal scholarly communication.

Preliminary findings of the Sense About Science 2009 Peer Review Survey were released earlier this month with some reaffirming results:

  • 91% of authors felt that peer review improved their last paper, with the discussion section receiving the greatest improvement
  • 84% felt that without peer review, there would be no control in scientific communication
  • Only 15% felt that formal peer review could be replaced by usage statistics

Still, the process is far from being perfect.

  • Only one-third believed that the current peer-review system is the best that could be achieved
  • One-fifth believed that the current system is unsustainable, chiefly because of too few willing reviewers

Anyone who reviews articles shouldn’t find these results surprising.  There is little training for reviewers, and they often receive little or no compensation for their work.  While there are benefits to knowing what your peers are working on and being able to influence the gatekeeping process, much of the time spent reviewing manuscript could be put into activities that result in greater rewards, like authorship.  Moreover, one often feels that providing a timely and comprehensive review is often rewarded with yet another request for review.

Improving participation in peer review is key. Just over half (52%) of survey respondents felt that some form of payment (e.g. a subscription or a waiver of their own publication costs) would make them more likely to review (41% even suggesting monetary compensation), although support for this option dropped to 2.5% if the author had to cover the cost.  Sound familiar?  Being acknowledged by the journal or receiving accreditation points (e.g., continuing medical education) were the most popular in-kind options.

The 2009 survey, based upon upon an earlier 2007 survey by Mark Ware and commissioned by the Publishing Research Consortium presents the results of 4,000 authors randomly selected from the ISI author database.  The results are statistically accurate within 1.5%.

Reblog this post [with Zemanta]

About Phil Davis

I am an independent researcher and publishing consultant specializing in the statistical analysis of citation, readership and survey data. I am a former postdoctoral researcher in science communication and former science librarian.


2 thoughts on “Peer Review Survey 2009

  1. Having managed medical/scientific peer review, none of this surprises me. I would, however, offer the following to balance your first paragraph, which characterizes peer review as “slow, inefficient,” etc.: Peer review also leads to stronger communication of findings and analysis, contributes to a sense of community, presents an important way to support one’s field, and increases confidence in the soundness of research that is published.

    As for the problem of the conscientious reviewer being asked to review again, in my experience a prompt “can’t do it this time” brings joy to the heart of the peer-review manager, who can then (a) quickly continue the search for a reviewer and (b) appreciate that this reviewer is willing to say no.

    The lack of tangible reward for reviewers is seen as more of a problem because of the sheer number of journals and journal submissions (which tend to increase also with the advent of online submission). An atmosphere of competition for reviewers leads to a longing for some kind of edge in this arena, but direct compensation of any kind can erode that key confidence factor. It’s no wonder that CME credits are an attractive alternative. And of course, acknowledgment is smart. But to lower the pressure for reviewers, it would help if every journal had a vigorous and careful rapid-rejection protocol in place such that all clearly substandard or inappropriate papers were rejected without review. This claims resources, like everything else, but the bottom line can be positive.

    Posted by Laurel Ferejohn | Sep 22, 2009, 10:59 am


  1. Pingback: Should Universities Pay for Peer Review? « The Scholarly Kitchen - Mar 16, 2011

The Scholarly Kitchen on Twitter

Find Posts by Category

Find Posts by Date

September 2009
« Aug   Oct »
The mission of the Society for Scholarly Publishing (SSP) is "[t]o advance scholarly publishing and communication, and the professional development of its members through education, collaboration, and networking." SSP established The Scholarly Kitchen blog in February 2008 to keep SSP members and interested parties aware of new developments in publishing.
The Scholarly Kitchen is a moderated and independent blog. Opinions on The Scholarly Kitchen are those of the authors. They are not necessarily those held by the Society for Scholarly Publishing nor by their respective employers.
%d bloggers like this: