I want to revisit the genus of portable peer review, a model in which peer review is conducted outside the auspices of the journal. We’ve covered portable peer review over the past several years, by interviewing its founders, exploring incentives and credit for peer review, and covering failed attempts.
This post will briefly focus on three companies that currently provide portable peer review: Rubriq, Axios Review, and Peerage of Science. I am conspicuously avoiding covering companies that manage the peer review process for publishers, consortia of journals that agree to transfer manuscripts and reviews, as well as cascading peer review within a single publisher. This is a scholarly publishing blog, so you are likely to find good coverage of these topics in other posts. And last, my purpose is not to provide an exhaustive review — you’ll find extensive details on each company’s website — but to provide you with the salient differences in how these three services conduct portable peer review.
Rubriq is built on a service model that matches manuscripts with registered reviewers. Since 2012, Rubriq has completed 1,688 article reviews. The company has little information on the fate of the papers that were reviewed, however. According to Damian Pattinson, VP of Publishing Innovation at Research Square, about half of the submissions were from authors themselves, the other half were from publishers. Pattinson does not know how many of the papers were re-reviewed by the publisher — either as a standard procedure, or because the publisher was conducting an evaluation of Rubriq quality — nor does he know the number of papers that have been published. Pattinson explained that Rubriq has been expanding its services for publishers, providing certification that manuscripts have been checked for plagiarism, figure manipulation, and quality of English, among other factors. Prices for reviews start at $600 USD, with reviewers receiving a flat fee of $100. Rubriq is a division of Research Square, located in Durham, North Carolina.
Axios Review is built on an agent model that reviews manuscripts, matches them with appropriate publication outlets, and contacts journal editors on the authors’ behalf. Because of their active role in the review and referral process, Axios has collected extensive data on their performance. According to Tim Vines, Managing Editor and Founder of Axios Review, Axios has reviewed 420 papers to date, 130 (31%) of which are known to have been published (see list). The remainder of the papers are in process of being revised by the authors or submitted to a journal. A small fraction of manuscripts have likely been abandoned, meaning the authors have stopped trying for publication. Of the 130 published papers, 65 (50%) were re-reviewed by the journal; the other 50% were accepted without further review. 85% of papers reviewed by Axios are accepted for publication by the first journal. Axios charges authors $250 USD which is due upon receipt of reviewer comments. Some publishers (chiefly BMC) deduct this fee from their article processing charge. Editors and reviewers are not paid for their work. Tim Vines reports that submission numbers have been growing at 30% per year. Axios Review is a registered non-profit company operating in British Columbia, Canada.
Peerage of Science is built on a community model, where authors are in charge of the peer review process. Unlike Rubriq and Axios, Peerage of Science does not solicit reviews on the authors’ behalf, but leaves the process up to their community of registered peers. As the author specifies the time period under which specific tasks must be accomplished, the number of manuscripts initially submitted to the system gets whittled down at each stage. According to Janne Seppänen, Co-Founder, and Managing Director of Peerage of Science, 364 manuscripts have been submitted to the system since 2012, 21% of which have failed to receive a single review. Authors can also terminate the review process at any stage, so the number of manuscripts that get reviewed, revised, received a final evaluation, and exported to a participating journal is about one-third (35%) of initially submitted manuscripts. The success rate drops to 28% of papers that receive a publishing offer or an invitation to submit directly to the journal. Peerage of Science posts a list of known published papers, although Janne does not know whether the journal put these papers through additional rounds of peer review. Their business model is supported by publishers, who pay €220 Euros for each peer review process leading to publication. Submissions since 2012 have been relatively constant, with an uptick in 2016. Peerage of Science is a for-profit company registered in Finland.
Portable peer review has had working models since 2012. Taken together, the number of manuscripts flowing through these services is tiny compared to the traditional model of managing peer review in-house. As detailed above, giving authors the freedom of managing or porting their reviews comes at the expense of gathering good data. Axios takes an active role in the review and referral process, while Peerage leaves most of the management details up to the author and reviewer community. Rubriq falls somewhere in the middle. Axios is supported by charging authors, while Peerage is supported by charging publishers. Rubriq appears to be shifting from author to publisher-side charges.
Among the three companies, only Rubriq reviews can be considered completely portable. Working on behalf of the author, Axios’ agent model limits author choices, although it appears to be highly effective in matching manuscripts with prospective journals. Similarly, Peerage of Science’s business model requires publishers to pay for review transfers, so portability is limited to a group of journals who have bought into the Peerage system.
Small markets coupled with the diversification of the leading company (Rubriq) into services aimed at supporting publishers may indicate that portable peer review has limited overall reach. Nevertheless, like book publishing agents, Axios’ agent model may have tapped into a real need to provide author services that do not stop with peer review. Peerage of Science, in contrast, gives authors the tools they need to be agents of their own manuscripts.
Journals have a history of outsourcing parts of its publication process to external service operations who can do it faster, cheaper, or better than keeping the process in-house. So, why hasn’t peer review been outsourced? There is far more diversity in how journals conduct peer review than, say, page composition or XML conversion. Perhaps it is this ecosystem diversity that has prevented portable peer review from taking off?