The theme of this year’s Peer Review Week is transparency in peer review. Many peer review experts will be gathering in Chicago in September for the Peer Review Congress (PRC), an international event that is held every four years. So we will be kicking off this year’s Peer Review Week celebrations with a panel session immediately after the Congress closes, at 5.30pm on September 12. Under the Microscope: Transparency in Peer Review, which I’m delighted to be chairing, will be open for all PRC attendees to join in person, as well as being live-streamed and recorded so others can also participate (register here for free).
To whet your appetites and encourage you to join us there or follow the proceedings online, we invited the four speakers to share their initial thoughts on what transparent peer review means to them and why it’s important. Irene Hames (independent peer review and publication ethics expert), Elizabeth Moylan (BioMed Central), Andrew Preston (Publons), and Carly Strasser (Gordon & Betty Moore Foundation) bring an interesting range of perspectives to the discussion, as you can see from their answers to this question. While they all agree on the importance of peer review, there’s divergence around what we mean by transparency in peer review and, critically, how to achieve it. For example, is there an agreed definition of what peer review actually is? Is it really the case that peer review has to be open in order for researchers to get credit for it? And how open should reviews for rejected papers be?
As moderator, I’m remaining neutral on the topic (at least for now!) but I look forward to your comments on this post and warmly invite you to submit additional questions for the panel either here in the comments or on Twitter, using the hashtag #AskPRW – and, of course, to join the discussion on September 12.
To me, transparency in peer review means making clear to everyone the how, what and why behind editorial decision-making. How are the peer-review and editorial processes organized at a journal/organization? Too often details aren’t given. What information was used in decision-making? Why was the decision to publish made? Currently, making the reviewer reports, editorial correspondence and author responses available with articles isn’t done by the majority of journals, so it’s mostly impossible to know what information forms the basis for decisions or to see the quality of the reviews on which these decisions are based. When I’ve reviewed manuscripts and then seen what the other reviewers have written I’ve been surprised, and disheartened, to see how brief and superficial their comments have sometimes been.
Trust has always been at the center of scholarly publication, but with the problem cases we’re seeing reported – Retraction Watch reports that more than 500 papers have been retracted because of fake peer review alone – confidence in peer review has taken a real knock. Opening up the process and enabling readers to see the content and quality of peer reviews will do a lot to help restore confidence.
Why can’t making reviewer reports and author responses available with articles become the norm?
Not only will this make things more transparent, it would be a great and simple way to address another big current problem – ‘predatory’ journals. Having peer-review reports alongside articles would go a long way to helping researchers distinguish between reputable journals and those that are predatory, questionable or carrying out inadequate peer review.
To some, transparency is synonymous with ‘open.’ At BMC, open peer review began on the medical journals in the BMC series in 1999, and open peer review is now commonly accepted across the industry as an important aspect of open science. Open peer review facilitates accountability and recognition, and may help in training early career researchers about the peer review process. Over 70 of our journals across various disciplines operate fully open peer review where the content of the invited reviewer’s report and name are shared alongside the published article (as others do e.g., BMJ, F1000Research).
For me, the term ‘transparent peer review’ has the precise meaning that the content of a reviewer report is posted alongside a final published article but no information about the reviewer identity is provided. Sharing the report content goes some way to making the peer review process more transparent and various journals operate this system, e.g., EMBO and Nature Communications. Others also practice another form of transparency, sharing the reviewer names but not revealing report content (e.g., Frontiers and Nature).
Peer review has historically been performed in silos and behind closed doors. We have little insight into who is doing the review or what issues are discussed during the review process. It can sometimes even be hard to know for sure that review is actually happening (this is the predatory publishing problem)!
This results in all sorts of problems, which – as Irene says – can undermine our ability to trust published research. It is also incredibly inefficient. To give an example, researchers who aren’t recognized for their time are less likely to prioritize review. This is compounded because editors have no insight into reviewer workload, meaning they can often overload reviewers with multiple review requests. The net results is a system where it often takes 10 or more invitations in order to get a manuscript reviewed by two reviewers.
To me, transparency is about making sure that we are able to reveal the information necessary to bring a level of trust and efficiency to the peer review process. It doesn’t necessarily require open review (although that would be nice), but it does mean coordinating across the publisher divide in order to give the community an understanding of who is shouldering the workload, what they have on their plates right now, and exposing information about the expertise of reviewers and the quality of the review process. As an industry we should be able to do this in a way that results in better, faster review and vastly increases our trust in published research.
Science is trending towards more openness. Open science practices have been gaining momentum for the last few years, especially with the advent of new tools like Jupyter Notebooks, increased uptake of open source software, and new publishing models that promote open access to all aspects of research. The idea of transparent peer review is increasingly discussed as a way of pushing science further into the open. Instead of reviews being cloaked in secrecy and seen only by editors and authors, open peer review allows the reader to see how the science unfolded. It would allow junior scientists to better understand the process of peer review, would result in more useful and thoughtful reviews (since the reviewer knows her/his review will be read by many), and can spark conversations and collaborations that might not have otherwise occurred. It would also permit credit for reviewers, which currently is a service that is performed without incentives or rewards.
With many thanks to Shane Canning of F1000 for his help in organizing the panelists’ contributions to this post