As a managing editor, one of the most common questions I get is about the journal’s acceptance rate. I’m typically puzzled by this because acceptance rate tells you very little about the likely fate of any one submission.
If all the submissions for a month went into a hat and a blindfolded editor pulled out a proportion of them to publish, the acceptance rate would indeed be a good indicator of an individual paper’s chances of publication. In reality, if a paper is below the quality threshold for the journal, it’s almost certain to be rejected; and if it’s above that threshold, then it’s almost certain to be accepted.
The interest in acceptance rate seems to be linked to the attitude that peer review is a coin toss, and hence the overall acceptance rate can predict the fate of each paper. Where does this attitude come from? Does it have any basis in reality?
Part of the answer comes from the binary nature of editorial decisions themselves — papers are ultimately either accepted for publication or rejected — whereas quality is a continuous variable. Imposing a binary outcome on a continuous variable leads to a situation nicely described by the logistic model. For the sake of argument, I’ve put together this logistic curve for Molecular Ecology papers decided in 2006. In that year, we accepted 332 papers and rejected 369.
The x axis is “quality,” as measured by the average reviewer score a paper received for its final decision. A “reject” recommendation is assigned a score of 1, “reject, encourage resubmission” is assigned 2, and “accept, minor revisions” gets a 4. These scores are then averaged across reviewers. (Molecular Ecology doesn’t offer reviewers the option of “accept, major revisions,” but this does exist as an editorial decision, which is why the score skips from 2 to 4.) The circles are the actual data points, but since many are superimposed, the approximate density can be seen in the ‘rug’ of vertical lines. The curve shows how the probability of acceptance changes with average reviewer score.
If peer review were a pure lottery, the curve would be flat, as all papers would have an equal chance of acceptance. However, there’s clearly a steep transition between an average reviewer score of 2.25 and 3.25 (corresponding roughly to acceptance probability increasing from 0.2 to 0.8). Only 25% of papers fall in this range; the remaining 75% are a clear “accept” or “reject.” Knowing the average acceptance rate for a journal is thus a hopeless predictor for the fate of a single paper — it all depends on how good it is relative to the journal’s standards.
A second part of the perception that peer review is a lottery stems from the disconnect between the needs of authors and the needs of journals. Authors need to publish their work in the highest impact journal possible, which, taken to the extreme, means that their paper should be the worst published by that journal that year. If it’s not the worst, then it’s possible it might have been accepted somewhere else even more prestigious. By contrast, journals need to maintain their current status by accepting only average or better-than-average papers, or otherwise their overall quality will decline. The end result is that authors will try to pick journals where their paper falls into the transition part of the figure — too far to the left, and they will almost certainly be rejected; too far to the right, and their paper will be easily accepted and should have been sent somewhere better. Authors following this strategy thus aim for a more lottery-like peer review process, a behavior apparently encouraged by directly rewarding authors for each publication.
Authors are generally careful about where they first submit. If not, the top journals would get swamped by millions of submissions each year. Journals also discourage the lottery approach by offering better service to authors with better quality papers, such as highlighting their research with “news and views” pieces or fast-track publication. These enticements help journals attract papers that are well to the right on the quality-acceptance figure. One could also view the multitude of submission and formatting requirements as a deterrent to submitting the same paper to a succession of different journals, although this is clearly less effective in the digital age. Authors mostly pay with their time.
At the end of the day, does it really matter that peer review is viewed by some to be a lottery? From a global standpoint the answer is unequivocally, “Yes.”
Authors who view the peer-review system as a lottery put unnecessary burden on the reviewer community as the same papers are reviewed over and over as they cascade down the hierarchy of journals.
The attitude that peer review is a lottery may also discourage authors from using reviewer comments to improve their paper before they resubmit elsewhere — after all, if acceptance is unrelated to quality, what’s the point? This behavior induces also weary cynicism in the community, as spending many hours reviewing a paper for one journal and then being asked to review an identical version for another induces a mix of rage and despair.
In summary, the peer review process can be a lottery if you want it to be, but the decision on most papers will depend strongly on quality (as adjudged by the reviewers). The real loser of the “lottery attitude” is the reviewer community, but what can be done to fix this problem is still up for debate.