Editor’s Note: Today’s post is by Avi Staiman. Avi is the founder and CEO of Academic Language Experts, a company dedicated to assisting academic scholars to prepare their research for publication and bring it to the world.
Peer review can be an opportunity for authors to rethink and refine their arguments. For example, vaccines developed by leading pharmaceutical companies to combat Coronavirus were scrutinized by scientists the world over using the peer review process to ensure the veracity of the research and to suggest tweaks and improvements. Trust in the scientific review process in general, and peer review specifically, was put to the test and much of the public debate hinged on the reliability and transparency of the expedited review process.
Peer review of academic articles is not without its numerous faults and it has come under attack on multiple fronts in recent years. Some critics point to peer review missing fundamental flaws in published papers, resulting in problematic research that requires retraction, while others take issue with the fact that peer review can lead to biased results. Additional issues include the fact that peer review tends to be slow and expensive with reviewers generally remaining unpaid. Furthermore, in many cases, reviewers are far less experienced researchers than the scholars they are reviewing.
These legitimate critiques generally fall short of recommending doing away with peer review altogether in lieu of a preferred alternative. As such, peer review continues to be the central process around which scientific knowledge is explored, critiqued, scrutinized, and refined.
In my experience as the CEO of an author services company, I encounter dozens of these reviews on a weekly basis and I am troubled by what I perceive as a steady decline in the pervading culture and attitude shown in these reviews, specifically for journal submissions. It seems that many reviewers see their primary role as deflating the arguments and methodologies of the manuscripts they receive, often without any concern for the way the author will receive the comments or whether the critique can be addressed and revised.
Rather than being an educational process that can help scholars improve their research and catch errors, peer review has become an arena where it is acceptable to critique and tear down others. For example, Adriana Romero-Olivares, professor of microbial ecology at New Mexico State University, details the demoralizing experience she had when her reviewers went into great detail to criticize the language and writing style of her article, without even bothering to address the underlying scientific findings. A Facebook group named ‘Reviewer 2 Must be Stopped!’ has tens of thousands of members who share their poor, and at times, outrageous review experiences.
Authors, meanwhile, are reverting into ‘fight or flight’ mode by responding in kind to reviewers or shelving the research altogether. Authors feel ‘battered’ rather than ‘bettered’ in their reviews and respond either by becoming intransigent in their approach or by beginning to doubt the quality of their own research and even themselves as researchers. These authors feel the sting of reviewers who dismiss or reject their manuscript, seemingly without evincing any responsibility to explain how they would have done things differently or what the author can do to improve their research in light of the critique.
Collateral damage of broken peer review
The anonymous review system makes it difficult to conduct constructive dialogue and authors are left feeling helpless and frustrated, not knowing what reviewers want from them or how they can improve their research. This is especially prevalent among more junior scholars who have less experience handling rejection and figuring out how to absorb and dissect the reviewer’s comments (on both a cognitive and emotional level). Sometimes simply trying to decipher reviewer comments can prove to be a herculean challenge. This is especially true for articles written by EAL (English as an Additional Language) authors whose methods and writing conventions likely differ from those of the reviewer.
Reviews that are especially mean or critical, without providing tangible points for improvement and growth, can lead scholars to bury their research on their laptops, not daring to send it off to another journal for a second attempt at publication. In an ironic twist, some of the world’s largest publishers, who are desperate to grow their journal collection as quickly as possible, end up turning scholars off, dissuading them from submitting and publishing their work due to the frustration of the review process.
It is of course important to qualify that reviews differ from scholar to scholar, and some reviewers do try and take an educational approach to help colleagues understand what they can do to improve their work. It is also important to note that some reviewers see their work as ‘volunteering’ and the amount of time they invest in thoughtful feedback may be a reflection in the degree of priority given to such reviews. Many journal editors take great care to sift through reviews and remove any demeaning or otherwise irrelevant critiques. However, the global reach of digital journals and the move towards open access has incentivized publishers to launch more and more journals and publish more and more articles as quickly as possible, putting editors under ever increasing pressure and limiting their ability to conduct proper oversight.
The critique-heavy approach extends beyond peer review and permeates many other areas of academic life as well. Research students quickly learn that scholars are expected to trade barbs with one another throughout their careers. Resulting feelings of loneliness and isolation that follow should come as no surprise. We must ask ourselves what culture and attitude critical review instills within us as scholars and whether it encourages us to be at our best, or perhaps inhibits us from growing as professionals and improving our work? Are we truly a ‘scholarly community’ or have we learned to pull ourselves up by dragging others down?
Constructive critique benefits science
To be clear: I do not propose making the peer review process any less rigorous, nor do I think that reviewers need to couch their critique in unnecessarily complimentary language to avoid hurting anyone’s feelings. Being able to receive feedback and criticism and knowing how to digest rejection is an essential part of the academic maturation process. In fact, research that is rejected can end up being more successful than other research that is accepted straight away.
I believe that peer review needs to extend beyond a formulaic system for sifting through and picking out the ‘best’ or ‘most innovative’ research from the rest of the pack. Rather, it should be seen as an educational process whereby reviewers help authors to sharpen their methods, refine their arguments, and consider alternative perspectives.
This view of peer review is not without precedent. The earliest known form of what we commonly refer to as peer review dates back to the 9th-century. Ishāq ibn ʻAlī al-Ruhāwī, author of Ethics of the Physician, proscribed that following patient visits, doctors should write notes that would then be reviewed by a council of medical professionals to determine the quality of care provided. The feedback would then be shared with the doctor in an attempt to help the doctor to reflect and improve.
So long as researchers retain their role as educators, it is our obligation and responsibility to not only ‘batter’ our peers but also give direction for ‘bettering’. In other words, peer review has the potential to be transformed from a toxic, destructive experience into a constructive one. But how can we go about doing that?
An educational approach to peer review
A format that might be appealing, not only to authors and reviewers but to publishers as well, takes a page out of the work of HSS book publishers and how they review manuscripts.
One of the main differences between the STEM journal and HSS book submission process is that book acquisitions editors get involved in the process before the manuscript is complete (and sometimes before there is a manuscript at all). This process starts at a relatively early stage in the writing process, creating a situation whereby editors are incentivized to help authors and sign them up before other publishers can swoop in and publish it themselves. Consider the potential parallels with the increasing use of journal preprints, as a place where journal editors could hop in and start the process of working with authors at an early stage in the process. (It may also help that you can submit book proposals to multiple publishers simultaneously).
If editors’ time was freed up to work with a cohort of scholars with strong underlying research that needed direction and focus, the resulting research could be vastly improved. Publishers looking to increase their research output could develop a reputation among authors as being helpful and constructive (words not generally used when discussing academic publishers), incentivizing authors to submit their research in a place where it would be given attention and care. This give and take could also help assuage scholars’ fears of making embarrassing mistakes and minimizing risk of future retraction. While each article may not receive the suggested 130 hours of professional expertise shown by some HSS journals, creating an editorial culture where the role of the editor is less about finding reasons to reject a manuscript, and more about finding ways to improve it would pay dividends to publishers in an era where author experience is an increasingly important differentiator between journals. Since business models are increasingly based on the volume of good papers a journal publishes, activities that can potentially increase submissions and the number of resulting high-quality papers published could potentially cover the associated costs of those extra efforts.
By making education and the improvement of the scholarly community the center of our peer reviews efforts, we would be doing a service not only to scholars on both sides of the review process but to the general public who funds and benefits from the resulting findings. If we can transform dialogue between author and reviewer into a constructive and positive one, it would help authors improve their research and give them a better shot at surviving the reviewers’ critique. If we reframe the peer review process for reviewers as helping authors improve, there might be less of a feeling of exploitation and more willingness to give serious time, thought, and consideration to their own suggestions. And finally (maybe most importantly), the resulting science would be improved giving people access to knowledge based more on facts and the scientific method and less on ego, pride, reputation and self-worth.
There would still be plenty of rejections from reviewers, but the role of the reviewer would primarily be suggesting constructive improvements and recommending publication, not identifying flaws or ad-hominem attacks. I think any journal that took such initiative and offered such service would quickly become a magnet for authors. Let’s reject the ‘battering’ peer review process and see if we can’t transform it into ‘bettering review’ instead.