While preparing for an EASE panel discussion for Peer Review Week 2021, I spent some time thinking about recognition in peer review.

Recognition in peer review is not just about recognizing the peer reviewers for their voluntary service to scholarly publishing and the research system as a whole, but it encompasses a few other aspects as well.

The first aspect of recognition is who is recognizing whom and how. Though the practice varies a lot, I strongly suspect that all readers of this article value peer review as an important means of validation of newly gathered knowledge. Selection of a reviewer by an editor is the first step of recognition. A ‘thank you’ note from the editor after submitting reviewer’s comments is a form of recognition increasingly becoming automated. Editors supporting reviewers’ comments while interacting with the authors or making a decision based on them is another form of recognition of reviewers’ expertise. When authors accept and address reviewers’ suggestions and perspectives in revised manuscripts, when they thank the reviewers through their response letter, or when they appreciate reviewers publicly in the ‘acknowledgements’ sections of their articles, even without knowing their names — we see peers recognizing peers’ valuable support.

many sticky notes reading "thank you"

However, being an anonymous reviewer, year-in-year-out, without any public recognition of our review work limits our abilities to contribute. With no public acknowledgement for our work at one journal, we won’t be appreciated by other editors or journals. If they approach us to review, it will be because of our other persona — an experienced author.

This is beginning to change. While almost all journals now have internal peer reviewer databases, many publicly list the names of their reviewers, some publish their review comments with names attached, many give certificates or offer incentives, like free access to subscription journals for a period of time. Publishing-related agencies are increasingly working on peer reviewer recognition. Publons, for example, supports peer-reviewer recognition by hosting reviewers’ profiles, linking with their review works, rating them, providing them with certificates, identifying them as mentors, offering awards, and collaborating with other agencies, such as ORCID.

However, we shouldn’t forget how peer reviewers themselves value their voluntary service and their association with certain journals, as they often mention such engagements in their CVs and public profiles, and on social media. Of course, there are numerous researchers all over the world who don’t make such public declarations or are not aware of recent developments noted above. They are just happy to do the review out of academic responsibility — we can call it ‘self-content recognition’. This used to be all peer reviewers in the pre-internet, pre-social media era. Are we losing that self-content part in peer-review recognition because of the system’s increasing pressures or our own hunger for public recognition?

The second aspect of recognition lies in the answers to a question: Is recognition in peer review a static thing, or does it lead to or transform into something else? For example, each act of peer review has an incremental effect in terms of recognition. When such recognition is publicly displayed, as profile badges or awards, it gradually contributes to building our overall reputation as individual researchers. Based on our timely delivery of high-quality reviews, we may also be invited to join editorial boards of journals — a new phase of recognition.

But, if our recognition as a peer reviewer is not converted into something tangible, can we still call it recognition? While the recruitment, promotion and tenure rules of our academic and research institutions vary a lot, they widely appreciate us writing peer-reviewed articles in journals. But our contributions to our respective disciplines through silent work, such as peer review, are difficult to include in these rules. Although not everybody is happy, some Dutch universities, for example, have recently banished the impact factor from hiring and promotion metrics. Others are exploring how to reduce focus on research publications and put more emphasis on academic activities, like teaching, and socially relevant and creative engagements. To put peer review together with the other outreach and community work academics often do, such as advising government and professional bodies, raising public awareness, engaging in debates, building professionals’ capacity, and mentoring the youth beyond home institutions, visibility seems to be an unavoidable step we have to take to be recognized. Being an anonymous, self-content peer reviewer isn’t enough anymore.

How recognition may look like in the changing peer-review landscape is the final aspect I want to talk about. Despite being around for almost three decades, in recent years, preprint servers are changing recognition in peer review by opening it up to ‘all’. While mainstream peer-review conversations, or debates for that matter, are still around single-, double-, or triple-anonymized peer review, preprints are making duplicate peer review not so unethical after all. Infrastructural advancements have made open peer review a more organized, credible community-based venture. Peer review is no longer strictly a pre-publication exercise. It can now be a part of post-publication workflow, even an action taken before data collection.

To overcome the challenges around peer reviewers’ availability, quality of review, and delays in the review process, other models, such as transferable peer review, creating reviewer pools by like-minded publishers, and AI-assisted peer review have been conceptualized, piloted, and practiced, along with improving the feedback literacy of reviewers. Approaches to peer review increasingly recognize diversity and inclusion, in terms of gender, race, age, and geography. Disruptive innovative concepts, like the Octopus publishing platform aim to communicate a piece of research as a chain of small pieces of publication, all subjected to open peer review by registered users. Each review would be ranked and visible on the reviewer’s profile page, which sounds like the ultimate recognition in peer review, with a healthy dose of reviewer accountability.

The way things are moving now, we may continue seeing peer review becoming a more open process, which encourages community engagement leading to a fluid reviewer identity and collective recognition. But is such a system exploitive? There are many reviewers who labor on behalf of journals that they cannot afford to read. It would be hard to imagine a similar transformation in other parts of scholarly publishing — would authors write papers as a community effort? Could journal editing become such a collective exercise? Can we picture the boundaries of competing journals being dissolved to serve the discipline as ‘one’? If the answers are “no”, then should we expect the future of peer review to remain a community venture, a crowdsourced process? Are we exploiting the altruism of peer reviewers by glorifying their roles without offering any tangible benefits for their services? Is “good karma” still enough of a motivation to do the time-consuming hard work of peer review?

Haseeb Irfanullah

Haseeb Irfanullah

Haseeb Irfanullah is a biologist-turned-development practitioner, and often introduces himself as a research enthusiast. Over the last two decades, Haseeb has worked for different international development organizations, academic institutions, donors, and the Government of Bangladesh in different capacities. Currently, he is an independent consultant on environment, climate change, and research systems.

Discussion

4 Thoughts on "Recognition in Peer Review"

Most academic institutions in the US do include peer review as a community-based activity that is supposed to contribute to tenure/promotion decisions. Whether it actually does is clearly up for debate (because let’s face it, not many universities implement policies fairly or objectively). I think a really interesting article would focus more on that – because as this article highlights really well, its not about peer review recognition, its about public recognition, which really ideally happens through tenure/promotion.

Thank you Kris, for highlighting how ‘recognition in peer review’ is in fact an issue that goes well beyond the boundary of scholarly publishing. While universities in many developed countries recognize community services, like peer review, in their tenure policies, many developing countries are still struggling to properly appreciate academic publishing itself in their university policies. In those countries, where journal publishing is largely a voluntary work done by academics for their societies or universities, I was wondering if an attempt to recognize community-based services in their tenure policies might be misinterpreted (by the authorities) or lead to malpractice (by the candidates).

Thanks for having focused the peer review recognition “in a very broad way”, as you said during the EASE panel at PRW: https://youtu.be/gZpB8SoVhAc?t=2546.

“It is so important that our system actually evaluate not only my teaching, the number of publications, the Impact factor of my Journals, but also the COMMUNITY SERVICE [please let me use capital letters to enlight the keyword], like being on this webinar should be recognized” 🙂

That’s definitely the point, and the peer reviewer contribution obviously plays a crucial role.

Thank you Angelo, for reemphasizing the need for recognizing our peer review service by our recruitment/promotion system. Without that the relationship between academic and scholarly publishing system doesn’t feel quite right.

Leave a Comment