The Institute of Physics (IOP) recently announced that it will be phasing in double-blind peer review for all of its journals, expecting the process to be complete by the end of next year. IOP is by no means the only publisher that has taken this step, but I have been interested to see what look like countervailing trends in the scholarly communication ecosystem in recent years: some entities moving toward more open peer review, and others toward maximally blind peer review. I posed some questions to Kim Eggleton, IOP’s Research Integrity and Inclusion Manager (@EggletonKim):

For those of our readers who may not be completely familiar with the nomenclature, can you explain how double-blind peer review works?

In double-blind peer review, the identities of the authors and reviewers are concealed from each other. In physics the traditional method of peer review has been single-blind, where the reviewers are aware of the authors’ identities. Double-blind is more common in humanities and social sciences.

As a term “blind” is ableist and very easily misunderstood – I’ve lost count of the number of times people have assumed “double-blind” means two reviewers. We are keen for the industry to move away from the use of ‘blind’ and look forward to more inclusive and clearer terminology being developed by the STM peer review taxonomy group.

What prompted IOP to take this step at this time?

We are always working on new ways to support and improve the peer review process at IOP Publishing. We’ve been offering double-blind peer review as an option on some of our journals for a few years now in response to demands from the community. Countless studies (Mulligan, Hall & Raphael, 2012; Chun-Man Ho, Mak, Tao, Lu, Day & Pan, 2013,  Jamali, Nicholas, Watkinson, Abrizah, Rodriguez-Bravo, Boukacem‐Zeghmour, Xu, Polezhaeva, Herman & Świgon, 2019) have shown that double-blind peer review is perceived to be the fairest model with the level of comfort for all parties, especially under-represented minorities increased in a double-blind scenario. After some analysis in 2018, we realized we needed to do more to make our editorial boards and reviewer pools as diverse as possible. We’re dedicated to ensuring that we provide a welcoming and equitable space for people that choose to work with us, whether that’s as an author, reviewer, board member or reader. As a learned society publisher we have a responsibility to do what we can to ensure equality, diversity, and inclusivity within science. Not just because diversity has been shown to benefit science, but because it’s the right thing to do.

Double-blind peer review is intended to improve both the quality of published research and the fairness and equity of publishing decisions. How do you see those factors interacting for you (and your authors)?

We believe that by concealing the name and institutions of authors, reviewers will be more objective in their reviews. Both conscious and unconscious bias have been proven to influence the rates of women being published for example, so we want to do whatever we can to reduce opportunities for bias. We survey all our authors at various points in the publication process, and we consistently see authors rating double-blind reviews as clearer, better quality, and more helpful than under the single-blind model – regardless of publishing outcome. Double-blind peer review isn’t just about getting more good papers out there, it’s also about reducing the number of bad papers. Having papers judged on merit alone, rather than author reputation (or lack of), can only be a good thing for quality and equality.

In recent years there has been a growing chorus of voices calling for open peer review, and there have been interesting experiments with varying degrees of openness, with varying degrees of success. What do you see as the differing roles of open and blind peer review? Can/should they coexist?

While there is an argument that all parties in peer review should be unblinded for maximum transparency, recent studies have shown that this method of “open” review is especially unpopular with early career scientists, who fear that more senior researchers might retaliate if they criticize their work. There is also evidence that suggests named reviewers are more cautious with their comments and don’t always give the constructive criticism they might have given if they were anonymous. Fully unblinded review also leaves more room for bias, from both the reviewers and the authors when responding to reviewer comments.

We will be offering double-blind review before publication, and transparent review (sharing the peer review reports on publication, and reviewers can choose to reveal their identity if they wish) post-publication. We believe these two processes complement each other perfectly, allowing for maximum objectivity during the review process, and maximum transparency after publication.

What costs and downsides do you expect this new policy to entail?

Moving to double-blind peer review does take time and resources, but we believe it’s worth it to improve equity of opportunity. We’re phasing the roll-out to ensure a smooth transition. Our plan is to have a number of IOP’s journals running exclusively double-blind review before the end of 2020, with the majority moving over in 2021. We’re thrilled with the response we’ve had from the community when we’ve discussed this idea, but we’re aware this move might not be popular with everyone. We just hope people can understand the motivations behind it and that its being done with the very best of intentions. Authors will be responsible for anonymizing their manuscript before submitting their paper and if authors would prefer to have their identity known to the reviewers, we will respect their wishes. We will still be expecting our reviewers to perform an unbiased, objective review, even if the authors declare their identity. If this policy helps just a few more papers be judged more objectively, it will have been worth it.

How will double-blind peer review work when so many submissions to your journals will already have been posted to arXiv?

As more authors post their preprints on servers like arXiv, some reviewers may be able to discover a paper’s authorship if they look for it. Anonymity isn’t guaranteed; however, there are studies published [Le Goues, Brun, Apel, Berger, Khurshid, Smaragdakis 2017; Snodgrass 2006]  that show reviewers often either can’t guess an author’s identity in a double-blind system or guess incorrectly. That’s supported by our own data too. A good reviewer won’t be purposefully trying to find the identity of the author anyway, they’ll be confident enough to judge a manuscript on merit alone.

How do you think this change will impact inclusivity in academia?

This is not a panacea. We’re one publisher operating amongst many in the physical sciences, and the problems of inequality in academia are systemic and deep-rooted. Academia is under increasing scrutiny for its lack of diversity and there are calls for action from many sectors of society. As publishers, it’s easy to say “it’s a pipeline issue” and pretend we’re free from influence. As gatekeepers of the products that (rightly or wrongly) are inextricably linked to incentive structures within academia, publishers have an influential role to play in making the system more inclusive. By doing our best to ensure peer review is objective, we can increase the proportions of under-represented groups that get published. That links to funding, to promotions, to participation on editorial boards – all of which bring about more diverse role models to inspire the next generation of researchers.  Publishers are part of the academic community and we need to accept our responsibility as part of this system, to ensure the status quo of inequity does not remain.