Note: In recognition of the role peer review has on scholarly communication, The Scholarly Kitchen will feature posts this week that celebrate Peer Review Week 2016. You can find other activities by following #PeerRevWk16 on Twitter or checking out the event listings at the Peer Review Week Website. 

Peer Review Week 2016 has officially launched with a focus on recognition for peer reviewers. I thought it would be a good time to take a view of the landscape of recognition.

The two hot topics when it comes to reviewers of peer reviewed scholarship are recognition and credit. These are two very different concepts. Credit implies that a benefit is being given. We talk about authorship credit whereby an author can point to a tangible benefit of being an author, or contributor, to a publication.

Recognition may lead to a benefit but is passive when it comes to benefits. The importance of recognition is that others see the recognition and any benefit is sort of happenstance.

Publishers have no influence for giving “credit” for reviews. This is entirely up to the crediting body — the reviewers’ institutions or funders. Journals could offer certificates or websites documenting the peer review contributions of an individual but the tangible credit or reward must come from someone capable of delivering it. Several weeks ago, I went on a hunt for any institution that “credits” employees for participating in peer review of either scholarship or grant applications. I didn’t find any.

A Wiley survey of over 3,000 reviewers found that “reviewers strongly believe that reviewing is inadequately acknowledged at present and should carry more weight in their institutions’ evaluation process.”

So here we are with a portion of the academic community asking for credit and the publishers or third party startups looking to provide credit despite the fact that the who might value the credit don’t seem to have an interest. If I am wrong and there are institutions or funders in serious discussions about giving researchers professional credit for performing peer review, do let me know in the comments. I would love to hear about it.

Whether an institution wants to credit people for peer review activities is sort of arbitrary from the publisher perspective at this point. The publishers are most definitely looking for ways to recognize peer reviewer contributions, and if that recognition leads to reviewers getting some sort of professional credit, that would be fantastic!

Since we aren’t the parties most capable of offering career advancement rewards, instead we need to ask ourselves, what can we offer that makes a difference? What do peer reviewers want most from journals? What keeps them coming back as reviewers? What kind of recognition motivates them most? We have several reports and surveys with some of these answers:

• Feedback: When asked what makes the reviewer most likely to review with a journal again, the top two answers in the Wiley survey were feedback from the journal on the usefulness or quality of the review and notification on the final decision made by the editor.
• Acknowledgment: Being named in a printed list of reviewers, reviewer of the year awards, and a certificates acknowledging review were also highly valued by respondents of the Wiley survey. Likewise, a Sense about Science survey found that 40% of respondents valued being publicly acknowledged by the journals.
• Payment in cash: There is a lot of controversy over paying reviewers. The “pro” argument states the value of the reviewer’s time, the fact that cash is almost always a good motivator, and that some publishers make a lot of money. The “cons” argument questions the conflicts inherent in a paid reviewer system. The Sense about Science survey found that 41% respondents favored cash payments for reviewing but that number dropped to 2.5% when told that the payment would be covered by the author. In contrast, the Wiley survey found that payments to peer reviewers were not a strong incentive for review.
• Payment in kind: The Wiley survey, the Sense about Science Survey, and a survey by Taylor & Francis all found that reviewers were favorable to receiving payments in kind such as discounts on Open Access (OA) fees, access to content, free color in their papers, and continuing education credits.

The payment option in cash or in kind carries administrative issues for publishers that really cannot be discounted. Reviewers have also stated in comments on blogs and on social media that the hassle of receiving payments that accumulate to an amount that requires tax forms to be filed is ridiculously counterproductive. Providing access to content is only valuable if the reviewer doesn’t already have access to that content, say through their university libraries. Discounts on OA fees or color charges seems more probable.

Moving on to the top rated incentives of feedback and acknowledgement, there is much that can be done. If you spend a few hours writing a review, you want to know if it is helpful. At a recent editorial board meeting for one of the ASCE journals, we talked about ways to make it easier for editors to personalize a thank you to reviewers and add comments about the review.

Journals should also send each reviewer the decision letters and associated comments to the authors. This is an important part of training. It also allows the reviewers to see the other reviews and try to analyze any differences.

Many journals also publish a list of reviewers each year. Going a step further, several organizations, including my own, have a program for recognizing Outstanding Reviewers. Editors select their best reviewers who then receive a certificate and a letter recognizing their contribution. We send the letter not only to the reviewer, but also their supervisor. A separate list of outstanding reviewers is published online. You can see examples of this here and here.

A somewhat controversial version of publicly acknowledging top reviewers was announced recently for the Journal of Behavioral and Experimental Economics. This pilot program will rate reviewers and then publish their names with their ratings. Those who rate really low will be spared the public shaming and, presumably, forego any public recognition for the contributions they may have made. The controversial part of this seems to be centered on the criteria for ranking, which is based entirely on speed of review.

Other publisher and third-party initiatives are aimed at provide a public glimpse into reviewer output.

Publons: Chances are you’ve been hearing about Publons for a while now. Straight up, Publons is a way for users to create a reviewer profile that includes all the journals they are affiliated with. Further, publishers can validate the review activity and depending on the journal policies, the reviewer may be permitted to list the papers reviewed and publish the reviews.

My Elsevier Reviews Profile: Elsevier has a new reviewer profile platform that also acknowledges the peer review activity of individuals. There is a certificate component as well as discount codes for Elsevier services. Reviewers can pull reports of their yearly activity, which may assist reviewers with US entry Visas that have a need to prove reviewer contributions.

ORCID: Publishers can use ORCIDs collected from reviewers to report their review activities on their ORCID profiles. How the publishers or agencies will transmit this information is a little fuzzy but “early adopters” include some of the major manuscript submission and review systems as well as Publons. The goal is to include all peer review activities, not just publications related.

Frontiers: This publisher has been publishing reviewer names alongside published papers but has now added reviewer activity to the Loop page (user profiles) on the platform.

Open Peer Review: There are journals that are publishing reviews and reviewer names alongside the published papers. F1000 Research, select BioMed Central journals, and The EMBO Journal (among others) are publishing reviewer names, which could be the ultimate form of recognition if this process works for the given community.

There is a flaw to some of these reviewer recognition initiatives. Papers under review are treated as confidential and as such, reviewers on a rejected paper will not qualify for any recognition tied to the publication of a paper. Outstanding reviewer programs and simple numbers (this person reviewed 5 papers for us) will provide recognition but will not be as granular and perhaps not seen as valuable. In establishing recognition programs, we must not forget the enormous contribution of the reviewers of declined papers.

As we celebrate Peer Review Week 2016, I am very interested in hearing from publishers what they are doing to recognize and incentivize reviewers; from institutions and funders whether they are moving toward credit for reviewers; and from reviewers what they want to get out of the system. I look forward to your ideas in the comments!