The theme of this year’s Peer Review Week — Identity in Peer Review — is critically important to scholarly peer review in all its forms, for article and book publications, for data, for grants, for hiring and tenure applications, and more. Posts this week will highlight issues of anonymous versus open review as well as how understanding, acknowledging, and addressing the many ways in which identity impacts peer review will help ensure more diversity, equity, inclusion, and accessibility.

We want to start by acknowledging the many implications and manifestations of identity in peer review. There are as many different identities as there are people in the world, but we also identify ourselves through membership of various groups — cultural, demographic, professional, and so on — that intersect in peer review (and across all forms of our professional experience in scholarly communications). In addition to considering the identity of individuals in their roles as authors, reviewers, editors, and more, we also need to think about how identity impacts the engagement among individuals throughout the peer review process.



There are big questions around the effect of open identification versus anonymous peer review. Does anonymity strengthen the review process, or does it weaken the author’s voice? Are there enough safeguards in place for open review to protect those in relatively weaker positions, such as early career researchers? There’s no doubt that there are times when knowing — or thinking you know — someone’s identity, and making assumptions about them and their abilities based on that, leads to biased, unhelpful, and unfair reviews. Adriana Romero Olivares, Principle Investigator for the Soil Microbial Ecology Lab at New Mexico State University, wrote about her experience of this in Review with Care:

“The authors need a native English-speaking co-author to thoroughly revise the grammar of this manuscript.” My heart sank when I received this feedback from a reviewer for the first paper I had ever submitted as a first author. The reviewer didn’t say a thing about the underlying science — but isn’t that what peer review is supposed to evaluate? Regardless, the reviewer recommended that our manuscript be rejected… I expected my paper to fare well. I knew the science was strong. I had the resources and backup data to make whatever scientific changes the reviewers suggested. I also knew that my English skills weren’t great. But I had a native English-speaking co-author and an adviser who was proficient in English, and they both thoroughly revised the manuscript… Had the reviewer assumed that, because my co-authors and I were all affiliated with the same Mexican research institute, none of us was proficient in English?”

And what about the issue of misrepresented identity — peer reviews supposedly written by a PI or other senior researcher but which are, in fact, written by their post-docs or graduate students? How can (and should) we make better use of the Contributor Role Taxonomy (CRediT) to identify and distinguish between internal and external reviews and/or between different types of review (of the data versus the article, for example, or of a video versus a book)?

There are also times when it’s important for everyone involved to know more, rather than less, about a peer reviewer’s identity. Patient peer review is a great example of this. While still not exactly common, a number of health science journals now encourage patients to review clinical and practice-based articles as “lay experts.” Clearly, while the identity of the patient themselves can, and probably should, be kept hidden, it’s critical for the authors to know that these reviewers are not clinical experts and, if appropriate, to have some context about how, where, and when they were treated, etc. Interestingly, at least some of the publishers that use patient reviewers have an open peer review process, publishing the review reports and requiring all reviewers to sign them.

Identity and recognition are also closely intertwined. Being recognized for something without revealing your identity is challenging, although there can be ways around this. For example, information in the review section of an ORCID record can be quite minimal. It must be added by a third party (typically the publisher or a service like Publons), making it verifiable and trustworthy, but it can include a date range rather than a publication date; a publisher rather than a specific journal; and so on. This information can then be shared in grant applications, and with promotion and tenure committees, so that the researcher can get credit for it, without sacrificing anonymity.

But the issue we most want to draw attention to as we launch into Peer Review Week is the ways that peer review is full of those who juggle multiple roles and identities — Editor, Board Member, Transgender, Black, Reviewer, Woman, Brother, Scientist — at once. And for some, separating their personal identity (e.g., woman) from their roles (e.g., scientist) has led to the development of safer spaces within the larger community. Examining where these overlapping positions intersect within the process of peer review presents several inherent challenges. For example, it’s not uncommon for editors to review for multiple journals or, in the case of editors, serve on more than one board at a time. Clear policies around this are needed because, as the CSE Editorial Policy Committee notes, “Having the same scholar as gatekeeper for manuscripts on any given subject area for more than one of the primary journal outlets in a field is unhealthy because it gives that person undue influence over what is being published in that field.”

For scholarly publications, the issues of identity in peer review have historically been tied to notions of expertise and status — both of which have been historically accorded to some groups of persons and not others. Those historical exclusions, particularly on the basis of gender and race, make it clear why identity in peer review is now so clearly connected to DEI efforts. The complexity of any individual’s identity offers both challenges and opportunities as we collectively seek more inclusive practice for better scholarship.

The features last week previewing Peer Review Week, Ask the Chefs and Ask the Peer Review Week Steering Committee, about the role of identity in peer review suggested some of the ways individuals and organizations are thinking about these challenges and opportunities. The posts this week also show us some of the efforts that organizations including publishers are undertaking to wrestle with them.

We would love to hear from more of you about your ideas and experiences of identity in/ and peer review.  Please share your comments here and on all the posts this week, as well as on social media.

Karin Wulf

Karin Wulf

Karin Wulf is Director of the Omohundro Institute of Early American History & Culture and Professor of History at the College of William & Mary. She is a scholar of early American and Atlantic history working on gender, family and sexuality.

Alice Meadows

Alice Meadows

Alice Meadows is NISO's Director of Community Engagement, responsible for engaging with and developing our member community. She was formerly Director of Communications and Director of Community Engagement at ORCID; and before that, she worked for many years in scholarly publishing, including at Wiley and at Blackwell Publishing. Alice is also a Co-Founder of the MoreBrains Cooperative, which provides consulting services to the open research/research infrastructure community.

Discussion

1 Thought on "Identity in/and Peer Review: Kicking off Peer Review Week 2021"

For many years, I adminstered double blind peer review. It’s possible that it won’t change much for established scholars — they could often guess the reviewers despite my attempts ar anonymisation. Now, I support people submitting articles in many fields. One clear issue is reviewers who concentrate on grammar rather than the science. These people tend to be in the USA, with fellow travellers in the UK. Another problem seems to be with very young, inexperienced reviewers, who tend to be unduly harsh. A final issue is with reviwers who append a long list of their own publications as ‘must reads’. It seems that knowing who you’re dealing with would help authors know how to react –but at the same time, our referees were often top people and very busy; they were usually ok with giving their time, but I don’t know if the same can be said of open peer review.

Comments are closed.