After many years with little to no formal support for new peer reviewers, there are now a lot of great resources — many developed by large publishers and societies, and most of them freely available. Sense About Science (a small, then UK-based not-for-profit) was one of the original leaders in the effort to provide peer review education for early career researchers. Starting with the publication of their widely-used I Don’t Know What To Believe booklet (first published in 2005), their program has grown to include regular (until COVID-19 in-person) workshops and other resources. Other more recently introduced examples include the American Chemical Society’s ACS Reviewer Lab, Elsevier’s Researcher Academy Certified Peer Reviewer Course, Springer Nature’s How to Peer Review tutorial, and the Wiley Reviewer Academy.
Last year, to coincide with Peer Review Week, IOP Publishing launched the first peer review certification program tailored for the physical sciences — an e-learning hub that forms part of their broader peer review excellence program. They are placing special emphasis on peer review ethics, which is a topic I have a strong personal interest in, so I’m grateful to their Reviewer Engagement Manager, Laura Feetham, for answering my questions about the program overall, and the ethics component in particular.
How and why did you develop your peer review e-learning hub?
Last year we carried out some extensive research into what motivates reviewers and gathered lots of information from early career researchers about how they felt the peer review process could be improved. The findings from our Peer Review Motivations Survey, along with the qualitative data that we collected from in-depth interviews with reviewers, indicated that there were three main areas for us to focus on: a call for more comprehensive and reliable training to help boost peer review competency; greater recognition for the work that they do; and better feedback from editors about the quality of reviews.
Our peer review training and certification program was launched to give researchers the ability to review with confidence. We started with a series of interactive online workshops which have been really popular. The introduction of our e-learning hub was the natural next step to take the program to an even broader pool of researchers. It means that all researchers in the physical sciences and allied fields can access valuable training at their own pace, from anywhere in the world.
It’s part of a broader peer review excellence program — what else does the program cover?
As well as providing training, we wanted to address the other needs highlighted by the peer review community. Most notably, we wanted to make sure reviewers received recognition when they submitted excellent review reports.
This led to the introduction of IOP Trusted Reviewer status — a certification which, once achieved, demonstrates an exceptional level of peer review competency. Our editors rate all of the reports we receive on a scale of 1 to 5, with IOP Trusted Reviewer status reserved for reviewers who submit a report rated 5 out of 5. Reviewers who have successfully passed our online training course or completed an interactive workshop are also given the option of reviewing a relevant manuscript for one of our journals. For those who graduate from the peer review excellence program the threshold for IOP Trusted Reviewer status is slightly lower, with certification being awarded for reviews that score 4 or 5 out of 5.
Another element of our peer review excellence program is providing reviewers with comprehensive feedback. In the current system, they rarely receive meaningful comments about the quality of their reports or to what extent their views shaped editorial decision-making. We’re developing a system that will enable feedback to be easily requested from our editors – so watch this space!
Is your peer review e-learning hub, and the program more widely, very specific to physics or are they also a resource for researchers in other disciplines?
The ability to write rigorous and constructive review reports is an art as well as a science, and most peer review skills are pertinent to all academic disciplines. However, our training is unique in that it is the only peer review certification program tailored specifically for the physical sciences. The modules have been designed to give the physical science community the ability to constructively critique scientific literature to an exceptional standard.
Your e-learning hub places a strong emphasis on publication ethics — can you tell us a bit more about that?
The increase in global scientific output is undeniably a good thing, but unfortunately it has also led to an increase in known forms of publication misconduct, and new forms of misconduct are emerging at pace. Peer review plays a crucial role in protecting the integrity and rigor of the scientific record and, as such, we think it’s really important that the next generation of reviewers have the skills and confidence to identify and call out publication misconduct when they suspect it.
So we placed a real emphasis on publication ethics when designing the training curriculum. Trainees learn about the main forms of misconduct, and tactics they can use to spot them when reviewing a manuscript. For example, trainees are shown examples of image manipulation and duplication to better enable them to spot this as reviewers, and to remind them to check images carefully. The training also covers data fabrication, and gives examples: trainees are encouraged to look out for datasets or figures that look ‘too neat’ or ‘too good to be true’, which is sometimes a sign that the data has been changed or fabricated. We aim to keep the training up to date as new forms of misconduct continue to emerge.
What sorts of ethical challenges do you typically come across in your peer review process, and how do you hope the hub will help address them?
The most common forms of misconduct we see relate to plagiarism. This includes self-plagiarism – where someone reuses their own words from previously published work — and authorship disputes, when we see researchers either intentionally missed off the author list, or authors added to the list when they don’t qualify. We also occasionally see allegations of data fabrication or unethically conducted research, as well as more emerging forms such as paper mills and image manipulation. The more we can identify these problems in peer review, the less the published literature is “polluted” and the more trustworthy and reliable the published record. Our hope is that, by educating reviewers on the types of misconduct and their various hallmarks, we can identify problematic work earlier and keep it out of the published literature.
What sort of usage are you seeing both for the program and, more specifically, for the hub?
We’re seeing high levels of interest and engagement with all our peer review training and certification offerings. Within a month of launching the e-learning hub, over 400 researchers from around the world had signed up to the course, with one quarter of them successfully graduating from the training.
Our online workshops are always fully attended and we will be running 20 of these during 2021.
We have also received great feedback from the community about our IOP Trusted Reviewer certification, with recipients noting that recognizing and celebrating quality peer review is really important. This initiative goes some way to addressing that. It was wonderful to see all the positive comments from reviewers across social media.
What are your goals for the program and the hub in the short and long term?
We’ve done lots this year, but there’s still much more to do! Our aim is to improve the peer review process for everyone involved – reviewers, authors, readers, and editors. In the short term, this will mean implementing a mechanism for reviewers to receive feedback on their reports, maximizing recognition and exposure for our hard-working and talented IOP Trusted Reviewers, and ensuring equitable global access to training in peer review.
Essentially, we want to engage and inspire the next generation of peer reviewers, thereby safeguarding the rigor of the research we publish.