When multiple scientists come up with similar solutions simultaneously, it may be time to take notice. In this week’s issue of Science, a letter to the editor, titled “Battling the Paper Glut,” proposed a solution to finding competent reviewers:
Journals should demand that for every paper submitted, an author provide three reviews of other manuscripts
If it sounds familiar, it is. The “golden rule” to peer-review was reviewed just last week in the Scholarly Kitchen in a fully formulated proposal to privatize the peer review system (see: “Privatizing Peer Review — The PubCred Proposal“)
In my review, I wrote that the PubCred bank solution was based on a tentative premise (that editors are experiencing a more difficult time finding competent reviewers), which is based on an even more tentative premise (that competent reviewers are overloaded with requests and others are unwilling to pull their fair share of the reviewing load).
It was time to hit the books and find out if these assumptions had any basis. Luckily there are a few peer-review experts out there and a number of exceptional studies:
- In a 2007 survey of three thousand academics conducted by Mark Ware for the Publishing Research Consortium, Ware reports that the most active reviewers were indeed overloaded. While 90% of responding authors claimed they were active reviewers — reviewing an average of eight manuscripts per year for three to four journals per year, and occasionally for an additional four journals — some reviewers clearly took on much more of the load. The most active reviewers claimed they reviewed 14 manuscripts per year on average, and 20% of requests were declined. Older and more senior academics claimed to review more papers than their younger colleagues.
- The 2009 Peer Review Survey of more than 4,000 authors conducted by Sense About Science reports that reviewers, on average, refuse to review only two manuscripts per year, with 39% accepting all requests and only 7% rejecting more than more than five requests. While there is clearly a group of reviewers rejecting many requests, the number of “overloaded” reviewers appear to be quite small. These statistics do not seem to suggest a “crisis” in peer review.
- Lastly, in a focused study of five biomedical journals (BMJ plus four specialist journals) on the reasons reviewers decline peer review requests, Leanne Tite and Sara Schroter reported that workload was the most frequent response to deny a request. Respondents were mixed on whether small financial incentives would encourage the acceptance rate, speed up the process, or result in a higher-quality review.
To boost participation and improve the timeliness of review, all three studies point to similar conclusions:
- Provide reviewers with free access to journal content
- Acknowledge reviewers periodically in the journal
- Provide reviewers with feedback on the outcome of the review decision
- Give reviewers feedback on the quality of their review
- Reward the best reviewers with appointments to the editorial board
Other incentives to reward reviewers also include waiving page or publication charges to reviewers or providing other in-kind forms of gratitude, such as a swell reception at a conference.
Reviewers generally felt uneasy with direct financial compensation for their time, especially if it requires authors paying the tab. Most academics view reviewing as part of their academic duties and not a form of moonlighting. Tapping into the academic reward system seems key to improving participation. As Tite and Schroter conclude in their study:
Reviewing should be formally recognised by academic institutions, and journals should formally, and perhaps publicly, acknowledge the contribution of their reviewers.