Trusting Blind Peers
Image by Gideon Burton via Flickr

Peer review is slow, inefficient, and prone to bias. It doesn’t always detect important work, is unable to identify errors and intentional fraud, and can be abused by editors and reviewers. Peer review is an expensive and imperfect process although it remains a highly desirable component of formal scholarly communication.

Preliminary findings of the Sense About Science 2009 Peer Review Survey were released earlier this month with some reaffirming results:

  • 91% of authors felt that peer review improved their last paper, with the discussion section receiving the greatest improvement
  • 84% felt that without peer review, there would be no control in scientific communication
  • Only 15% felt that formal peer review could be replaced by usage statistics

Still, the process is far from being perfect.

  • Only one-third believed that the current peer-review system is the best that could be achieved
  • One-fifth believed that the current system is unsustainable, chiefly because of too few willing reviewers

Anyone who reviews articles shouldn’t find these results surprising.  There is little training for reviewers, and they often receive little or no compensation for their work.  While there are benefits to knowing what your peers are working on and being able to influence the gatekeeping process, much of the time spent reviewing manuscript could be put into activities that result in greater rewards, like authorship.  Moreover, one often feels that providing a timely and comprehensive review is often rewarded with yet another request for review.

Improving participation in peer review is key. Just over half (52%) of survey respondents felt that some form of payment (e.g. a subscription or a waiver of their own publication costs) would make them more likely to review (41% even suggesting monetary compensation), although support for this option dropped to 2.5% if the author had to cover the cost.  Sound familiar?  Being acknowledged by the journal or receiving accreditation points (e.g., continuing medical education) were the most popular in-kind options.

The 2009 survey, based upon upon an earlier 2007 survey by Mark Ware and commissioned by the Publishing Research Consortium presents the results of 4,000 authors randomly selected from the ISI author database.  The results are statistically accurate within 1.5%.

Reblog this post [with Zemanta]
Phil Davis

Phil Davis

Phil Davis is a publishing consultant specializing in the statistical analysis of citation, readership, publication and survey data. He has a Ph.D. in science communication from Cornell University (2010), extensive experience as a science librarian (1995-2006) and was trained as a life scientist.


2 Thoughts on "Peer Review Survey 2009"

Having managed medical/scientific peer review, none of this surprises me. I would, however, offer the following to balance your first paragraph, which characterizes peer review as “slow, inefficient,” etc.: Peer review also leads to stronger communication of findings and analysis, contributes to a sense of community, presents an important way to support one’s field, and increases confidence in the soundness of research that is published.

As for the problem of the conscientious reviewer being asked to review again, in my experience a prompt “can’t do it this time” brings joy to the heart of the peer-review manager, who can then (a) quickly continue the search for a reviewer and (b) appreciate that this reviewer is willing to say no.

The lack of tangible reward for reviewers is seen as more of a problem because of the sheer number of journals and journal submissions (which tend to increase also with the advent of online submission). An atmosphere of competition for reviewers leads to a longing for some kind of edge in this arena, but direct compensation of any kind can erode that key confidence factor. It’s no wonder that CME credits are an attractive alternative. And of course, acknowledgment is smart. But to lower the pressure for reviewers, it would help if every journal had a vigorous and careful rapid-rejection protocol in place such that all clearly substandard or inappropriate papers were rejected without review. This claims resources, like everything else, but the bottom line can be positive.

Comments are closed.