Why does peer review take so long? This is a common complaint publishers hear from authors. There are many answers; most commonly papers are delayed because it is difficult to get enough people to agree to perform the review or reviewers fail to get their reviews done in a timely manner. There remains something of a disconnect for researchers between their expectations as authors and their behavior as reviewers (see also: complaints about reviewers asking for additional experiments). A fast review process provides a competitive advantage for a journal, and the Nature Publishing Group (NPG) recently performed an experiment in outsourced, fast track peer review to try to address the problem. The unexpected consequence of this experiment exposed something else entirely: concern from the research community over tiered systems where wealthy authors are favored over those lacking funds.
NPG launched a four-week trial in their megajournal Scientific Reports. The journal features a Gold open access (OA) business model, where accepted authors pay a $1,495 article processing charge (APC). In the trial, authors willing to pay an additional $750 upfront would get their decision in three weeks. NPG would be able to offer this additional speed by outsourcing the peer review process to Rubriq, a service offered by the private company Research Square.
This experiment offered an interesting opportunity to explore what’s been dubbed the “freemium” model as a way of funding OA journals. When we discuss the many things that publishers provide, we hear from some commenters that some of these services seem unnecessary, and often a proposed solution is that authors should only pay for the services they want. Taken to an extreme, one would decouple the various services from the journal, offer something like a preprint archive (e.g., arXiv) and authors would then pay separately for each service they wish to add to their paper (peer review, copyediting, marketing, etc.).
NPG’s experiment seems a first step in this direction. Keep costs low for all authors and let those for whom speed matters pay a premium. There is clearly demand for the offering, as some 25 authors elected to pay the extra cost for fast track peer review. However, the reaction from the research community as a whole suggests that the freemium model is unlikely to see broad acceptance in this manner.
As the program was announced, UCL professor and Scientific Reports editor Mark Maslin resigned from the journal in protest. This was quickly followed by other editors following suit, and eventually a protest letter from a group of editors was written, announcing their concerns. The general consensus was that this experiment set a dangerous precedent and would result in a two-tiered system, where one group of well-funded authors would receive different treatment from those lacking funds.
This gets to an inherent flaw in author-pays business models. One of the core goals of OA is to do away with economic barriers preventing researchers from accessing the literature. But by shifting the economic burden from the readers to the author, different economic barriers spring up, preventing those who can’t afford APCs from publishing their own work.
I’m in the midst of starting a Gold OA journal for a broad, interdisciplinary research society. The strongest resistance to the new journal has come from the society’s social sciences and humanities researcher members. Most don’t have funding, so the typical APC charge for such a journal is beyond their means. This leaves them feeling left out and expecting that the journal will end up as yet another service aimed at their wealthier biomedical research colleagues.
To combat this, we’re offering a generous waiver program. But the success of such programs can be somewhat variable and it is unclear whether they are broadly sustainable in the long term. There still remain no clear cut criteria for providing waivers or discounted fees for authors. PLOS has long been a leader in exploring these areas, and their program has evolved over time from “no questions asked” to a careful examination of an author’s financial portfolio. The additional scrutiny has not been well-received by researchers.
Without clear criteria and an ability to confirm that an applicant meets those criteria, a waiver program is difficult, if not impossible to administer. Putting together financial records for examination adds a time burden to authors; going through those records a time and cost burden to a journal, with those costs likely passed on to paying authors. In the case of NPG’s fast track, the time required for a waiver review would likely wipe out any advantage offered by the program anyway.
There is a deep philosophical egalitarianism that runs through academia–your success should come from the merit of your work, not from the size of your bank account. This runs contrary to systems where your treatment is based on your ability to pay. I worry that any sort of freemium approach, where finances so clearly permit an advantage to one group over another may never be acceptable to researchers.
The other really interesting thing to emerge from this experiment is a glimpse at the importance of brands to researchers, and whether outsourced peer review is something they’d willingly accept in an established journal. A key set of questions asked by the protesting editors was:
Who is going to choose reviewers, will it be colleagues or the company managers? Who will be ensuring that the rigor and standards of peer review are maintained? What about handling appeals and possible conflicts of interest? What about the expertise of the reviewers?
When an author submits a paper to a particular journal, they are doing so under the assumption that the process of review will be expertly managed by the editors of that journal. Each journal has its own carefully cultivated reputation. If you do research you likely know the “personality” of each journal in your particular field. Readers of journal articles similarly rely on journal brands as filtering mechanisms for decisions about what to read. If instead the editorial process is outsourced to a third party, does that change the nature (no pun intended) of the journal?
NPG notes that they, “conducted a successful private parallel run of peer-review outputs comparing Rubriq with Scientific Reports,” essentially noting that the quality of reviews received was similar between the private company and NPG’s own efforts. They may have a point for a journal that bases acceptance solely on the article being “technically sound and scientifically valid,” rather than on judgments of novelty or importance. If the peer review process is essentially a fact check, then the levels of expertise needed may indeed be less than a process that is meant to be an assessment of the quality of the work by a panel of experts.
I’m not sure outsourcing peer review would extend well to NPG’s other journals with different criteria for acceptance. Would you still read a journal the same way if the articles were selected by an anonymous group of randos rather than a publicly declared editorial board? Would knowing that you had no input into selecting peer reviewers make you less likely to sign on as an editorial board member? Would you feel you had sufficient information to make editorial decisions if you didn’t have a hand in selecting just the right peer reviewers?
Idealism aside, financial disparities are a fact of life in a system where limited funds are unevenly distributed. The lab that can afford the newest cutting edge equipment has an advantage over the lab that has to make do with older tools. A big, well-funded research group that can throw lots of investigators at a question likely has an advantage over one researcher working part-time on the problem.
Additional fees for upgrades are nothing new in the world of journals. One could easily view color charges or hybrid OA options as freemium offerings. Charging fees for a fast track review process may be seen as a step too far though, as it occurs before the accept/reject decision is made and thus enters into a realm viewed as sacrosanct. To be fair to NPG, fast track review for a fee is not something they invented and has been part of the publishing landscape for some time (a complaint about favoring rich authors dating to 2011 can be found here).
Perhaps the problem here is less in the concept and more in the execution. Adding outsourced peer review to an existing journal with a particular reputation for quality and an editorial board used to exerting control over the peer review process may be too fundamental a change. Similarly, by framing the fast track program as an add-on to an existing process, it sets up an uneven playing field for authors within the same journal. Had NPG done the exact same thing but set this up as a new, separate journal (Scientific Reports Express) with its own (outsourced and anonymous) editorial board and its own higher APC (with the $750 premium built-in rather than as an add-on), would anyone have protested?
Discussion
21 Thoughts on "Thumbs Down for the Freemium Model? Researchers Reject Nature’s Fast Track Peer Review Experiment"
As a former EIC, let me say any journal that guarantees a decision within X weeks must be run by clueless weasels. If everything goes right — good paper, on-the-ball reviewers, no major adverse issues — reviews can be done quickly. However, stuff happens, and the EIC always must be able to say, “We will work on this until we get it right.” Even good science sometimes needs the tender loving care or an expert reviewer well known to the editors, even if that means waiting.
Peer review is not an automated process where one just checks the boxes and makes sure facts are correct. It is an expert evaluation of the suitability of science to be published. It is not a majority vote, but a consensus the paper is worthy and, to the reviewers’ knowledge, contains no major flaws. The editorial board plays a critical role in making sure a thoughtful evaluation actually happens. Time deadlines come in second.
I think the heart of the problem is a growing disconnect between scholarly journals and the communities they support. It used to be that journals grew out of the community of scholars in their field mainly through societies. Researcher/scholars felt a connection to them and were willing to support them with the time and effort to review manuscripts and serve on editorial boards.
I started Medical Education Online in 1996 and edited the journal with Ann Frye for 12 years. We didn’t never had a problem finding reviews. We usually had a pool of about 300 medical educators that volunteered ahead of time to review and we drew from those being careful not to ask any individual reviewer more than once or twice a year. When we ran low, we put out a request for volunteers on the DR-ED listserv run by my colleague Brian Mavis which has about 1,500 followers who work in medical education. We would usually get about 40 or 50 new volunteers because many knew either Ann or I or at least heard of us. We were part of the medical education community and I think most thought of the journal as part of the community as well.
Now like most researchers with some publications I get asked over and over again to review by journals I have never heard of and have no connection. Much of the problem is from sleazy OA journals but far from all. The reputable OA and traditionally subscription publishers are also creating journals right and left and while they may take the trouble of getting a reputable editorial board and editor it’s still a “top down” approach that results in journals with no real connection to the community of scholars they serve. I have friends who have served as editors on some of these journals and they have all quit in frustration. They send out 20 requests to review and half the time no one responds.
It’s interesting that PeerJ doesn’t seem to have a problem turning around manuscripts in about 3 weeks and in my experience as an author and academic editor they do an excellent job of peer review. They are successful because there is a large community of researchers who believe in the model and willing to support it by putting some effort into both acting as academic editors and reviewing manuscripts.
I would suggest that what you’re seeing is not any sort of philosophical change from publishers nor a change in business models, but instead is a result of volume. The number of people doing research continues to rapidly increase, and the academic career system is built on continuous pressure to publish, publish, publish. As a result, there are more articles to review and more journals to publish those articles. The publisher you see starting new journals right and left are doing so because the demand exists for those journals. And as you note, good researchers with strong track records are bearing the brunt of the peer review requests coming out of all this volume.
I would also suggest that PeerJ remains able to turn things around so quickly because they’re dealing with small volumes of papers. It’s much easier for a journal to get a small number of papers through the review process in a timely manner than it is for a journal that is buried in submissions. As an example, note how fast PLOS ONE used to be–it built much of its reputation on the speed of its review process. If you look at what they’re publishing now, things seem to take as long or longer than most other journals, and I would bet that much of this is due to scaling problems.
I don’t disagree nor did I mean to imply it’s the fault of the publishers. I think it is due to the very rapid growth of science and scientific funding over the last 50 years and a the move away from society based publishing. Commercial publishers moved into scholarly publishing during the last half of the 20th century because it became profitable with the increase in research funding AND because societies could not keep up with the need for new journals. They filled a need. A byproduct however is a growing lack of connection between researchers and journals. It takes a lot of time and effort to review. Researchers are much more willing to review if they feel a connection to the journal.
I also agree PeerJ has been able been able to maintain quick efficient reviews because it is relatively small but also because the editors put a real focus on fast reviews and they have been able to create a sense of community through their very unique membership model. There are a lot of researchers, myself included, who are willing to put in some extra effort to support PeerJ because we believe in the model.
I think this issue was less about freemium publishing and more about who has authority to make policy decisions about how a journal is run. Had the editorial board been involved early in the fast track “experiment,” they would have likely supported and defended the study. Instead, editors felt they had no control over the editorial and peer review process or ownership in the decision-making process. The end-result (revolt and resignation) is pretty typical. I’m surprised NPG didn’t see this coming. It’s a story that seems to repeat itself regularly in scholarly publishing.
I agree that “ownership” issues come into play here. It would likely be much more pronounced for a society-owned journal than this one, which is owned by a private company. But editors were brought in under specific conditions and led to expect to play a particular role in running the journal. Pulling the rug out from under those editors played a key role in the reaction, and as noted, if this was an entirely new venture without those expectations in place, I sincerely doubt that any ruckus would have been raised.
I agree that editor buy-in is important. This could have been a quiet experiment to see if authors liked the service if NPG had included the editors in the decision. That said, there are literally hundreds of people on the board of Scientific Reports. I doubt they would ever get consensus on anything. This becomes one of the downfalls of the mega-journals. The editors in charge of selecting content are too far removed from the business, scope, and policies of the journal.
In what sense is this a “freemium” model? Where is the free tier? It seems like a “premium” model without a free dimension. Not all upgrades are from free to paid. Most are from paid to a higher level of paid.
On the point about waivers, why don’t journal publishers agree on a common financial reporting model like FAFSA for college applicants so that everyone has access to the same set of financial information about authors as they do about families with kids applying for financial aid?
These are such hugely important issues. Back in 2012 the American Historical Association rejected OA gold and other mechanisms that would differentiate among scholars with differing ability to pay. http://blog.historians.org/2012/09/aha-statement-on-scholarly-journal-publishing/
It seems to me that this is off-mission. Is the goal of the AHA to further historical research or to effect social justice? Those are not the same thing. Must *everything* be politicized?
When the situation has already been politicized and is directly threatening one’s existence, is it off mission to attempt to join the debate and have one’s voice and legitimate concerns heard? Or would the AHA be better off sitting idly by while scientists with millions of dollars in grant funding decide how things should work for historians with no grant funding?
One of the very early observations of the APC-based model of OA was that the financial incentives for accepting MS’s were inverted. The work of reviewing and rejecting weaker papers is supported only by the papers that are accepted. A high rejection rate becomes unsustainable.
In a “pay-for-review” the journal can earn money from accepted papers, AND from rejected papers.
NPG and RS have good, established, well-deserved reputations, but I would expect predatory peer review to arrive in this new ecological niche.
[Aside – it would be interesting to know how many of the 25 papers that paid for the Fast-track were rejected?]
Is anyone moving to a submission fee for all papers submitted rather than a processing charge for those accepted. For those with long memories it was much discussed in the early days of BMC but even Vitek Tracz did not try it out (I think). Might NPG try this?
Anthony
I don’t know about NPG, but submission fees seem the obvious answer to the question of how to do a highly selective open access journal. If the problem is that the few accepted articles have to pay for the hordes of rejected articles, then a submission fee drastically cuts down on the number of submissions, and those that remain pay for themselves. The problem is that very few journals are brave enough to take this step. It comes with a competitive disadvantage–when you’re in a competitive field and an author has a choice of journals, it’s likely they’ll go to the one without the submission fees.
That said, there are journals that have had positive experiences with them. A good example is the Journal of Bone & Joint Surgery:
http://jbjs.org/instructions-for-authors#SubmissionFee
“Would you feel you had sufficient information to make editorial decisions if you didn’t have a hand in selecting just the right peer reviewers?” Ha – this doesn’t happen at many journals anyway! How many scientists, or academics from a range of disciplines, have had peer reviews from reviewers who clearly know *nothing* about a paper? I just had a review back that started “I know very little about this topic…” wile other reviewers fail to recognise well-known terminology or theoretical paradigms. I’m not sure private peer review could be much worse; at least if you’d paid for the review, you could send it back and demand to have your paper reviewed by someone who actually understood the paper!