It’s not everyday that someone introduces something novel in scholarly publishing. Most new ventures are variations on well-established models. With the creation of Rubriq, co-founders Shashi Mudunuri and Keith Collier have broken new ground. Rubriq is an attempt to provide peer-review independent from journals. While others, such as Faculty of 1000, have done this with post-publication reviews, Rubriq’s model focuses on pre-publication – indeed pre-submission (to a journal) review.
Collier publicly unveiled Rubriq, now in beta release, at the STM Innovations Seminar in London in December (you can download his presentation here). Collier is not new to either scholarly publishing or managing peer review systems. Prior to co-founding Rubriq, he was Vice-President and General Manager of Thomson Reuter’s ScholarOne.
Unfortunately for Collier, he lives in proximity to this author and was therefore ambushed with a request for an interview over lunch. Collier’s responses were graciously provided via e-mail.
Q: What is Rubriq?
A: Rubriq is a for-benefit business with the mission of putting time back into science. We are developing a model for independent and standardized peer review. It’s important to know we aren’t a publisher and have no plans of becoming one. We provide rigorous reviews by the same qualified peers who review for journals, but with a standardized scorecard that can be used in any publishing model.
The core of our innovation is the scorecard itself as a quantitative peer review instrument. Our thinking is that a paper should be evaluated against a standard outside the lens of any single journal. Rubriq asks “How good is this article?” rather than “Should this article be published in journal X?” This is a big shift for reviewers. We believe that this shift will create a more consistent, transparent peer review process that can be used by any journal within the field.
Our system will enable fast, rigorous, verifiable, portable reviews, and will present authors with the best options for publishing their research based on the independent review.
Q: Who is Rubriq?
A: Rubriq is a division of Research Square, a privately held company created by entrepreneur Shashi Mudunuri, who also started American Journal Experts in 2004, and who collaborated with Keith Collier to co-found Rubriq.
Q: What problem did you set out to solve?
A: Rubriq can address many problems, but there are two specific issues we are focused on. The first is the individual pain felt by authors as they go through the publication process – too slow, redundant, inconsistent, biased – the list goes on.
Second is the overall amount of time spent on redundant peer review. Our calculations put time spent just on rejected reviews that are not shared with subsequent journals in the range of 11-15 million hours annually, which is staggering. We want to put some of that time back into science.
Q: How does it work?
A: There are three steps to the Rubriq Process.
Step 1: Compliance Check. We classify the paper by subject area and keywords, check for conflicts of interest and ethical statements, and run the paper through iThenticate. These combined items create the compliance check portion of the Rubriq Report.
Step 2: Peer Review. We match the paper to three reviewers who complete independent, double-blinded reviews of the manuscript by filling out the Rubriq scorecard and leaving bulleted comments for the authors. These reviewers receive ongoing feedback about their reviews, as well as compensation for each review completed.
The reviews are compiled into a single report—a Rubriq scorecard—with numerical scores for the quality of the research, the quality of the presentation, and the novelty and interest. These scores are used to calculate an overall score for the manuscript: the R-Score.
Step 3: Journal Recommendation. Based on author preferences, journal classification, and Rubriq Report scores, we create a list of journals that are matches for the paper. The journal list includes detailed information about each journal, including acceptance rates, time to first decision, and APCs, if applicable.
The author can then choose to submit directly to a journal, revise the manuscript based on the feedback, or share the report with journals in the Rubriq network to determine interest. Of course, they could also publish themselves directly to a repository.
Journals with Rubriq accounts will be able to search for new papers that match their preferences in terms of scores, topics, keywords, etc. They will also be able to set up alerts when new papers that match their needs are added to the system, and will be able to control all contact preferences with authors.
Q: What is an R-Score?
A: The R-Score combines the quality of the manuscript with its potential impact. Reviewers determine the level of novelty and interest, which sets the R-Score range, and the overall quality assessment determines how high within that range the overall R-Score will be.
Q: Are reviews single- or double-blinded?
A: Reviews are double-blinded in Rubriq. However, journal account holders will be able to see reviewer names and affiliations with author permission.
Q: What is the business model?
A: We charge a flat fee per submission and are currently operating on an author-pays model. In our discussions with institutions and funders, new models are being suggested to us that we are working to incorporate into the system. If grant funds allow for publishing services, the funder would essentially be the customer. If article processing charges (APC) can be divided between an open access (OA) publisher and Rubriq, that would remove some of the burden from the author as the fee could be applied directly to the publication.
Q: How much do authors pay?
A: Currently in our beta release, the cost to authors is $500 (US) for the scorecard. Pricing for the complete service (journal matching, iThenticate report and compliance check, and ability to share scores with journals in the network) is anticipated at $700. A significant portion of this fee goes directly to the reviewers to incentivize them to prioritize the review and return it within a few days. As a for-benefit business, our goal is to keep this cost as low as possible.
Q: Why would an author want to pay? What is the benefit to authors?
A: The idea behind Rubriq is very different from submitting to just one journal and getting a review specific to that journal. We feel if we can provide a Rubriq Report in about a week with data driven journal recommendations, we can save researchers weeks to months. In addition to feedback on the paper, they have a portable review and are presented with realistic options for publication.
Funders have indicated that they would like to see a splitting of the APCs across Rubriq and gold OA publishers. So in that scenario, where the journal offers a discount for papers submitted with a Rubriq Report, getting the report doesn’t cost authors any more than it would have had they submitted directly to the journal.
With Rubriq, authors can also see all their choices for publication before deciding where to submit or publish. They can compare journals more thoughtfully before making a decision. Since the author pays, they also retain control over their score and how it is used. They can choose to keep it confidential, and can choose when and where to share it – inside or outside the Rubriq system.
Q: Can/do institutions (universities or funding agencies) pay on behalf of authors? In other words, do you have an institutional model?
A: There has been interest from institutions in subsidizing Rubriq fees for researchers, and we can definitely support managing accounts from shared funds.
In addition, we’ve had promising discussions with Funders, and they see the system-wide economic benefit of Rubriq and how we can help researchers publish faster. We hope to work with funders and create a model that works for all parties.
We are also working on an institutional partnership model where funds earned by Rubriq reviewers from a university or lab can be pooled in a central account for others in that institution to use for their own Rubriq Reports.
Q: Can anyone access the Rubriq database and see the scores?
A: No. Authors have complete control over whether they wish to share their scores. All Rubriq Reports come with a verification code that can be used by third parties to access reports shared by the author. If an author chooses to share their reviews with journals in the Rubriq network, only journals with verified accounts can access the database of papers and scores. If an author wishes to only show their Rubriq Report to one journal, they have that control.
Q: Do publishers “subscribe” to Rubriq? How does the publisher relationship work?
A: There are no costs to publishers in our model, unless they are submitting papers. Journals can create accounts and set up search alerts for papers that are broadcast into the Rubriq system. We do want to indicate the journals that accept or require Rubriq reviews, so journals in our network will be highlighted on the Journal Recommendation Report.
Q: What publishers are you working with?
A: We have been pleasantly surprised with the positive response from the publishing community as they look for ways to improve the process, not just for their editors, but also for reviewers and authors. During this initial beta period, we are working with five publishers that range in size and model to help us test the system: a large commercial publisher, a mid-size commercial publisher, a society publisher, and two open access publishers. We can likely share their names in the coming weeks.
Q: Can publishers publish the R-Score?
A: We see the R-Score as a new article level metric that indicates the quality and interest of a paper based on three independent reviews. It is a starting point for post-publication comments and ratings. It allows an initial stratification of the literature, especially in the mega-OA journals.
Q: Can publishers publish the reviews from Rubriq along with the papers, much as eLife is now publishing artifacts from their review process.
A: It is important to know that if you are reviewing for Rubriq, you are essentially reviewing for any journal. And while we are performing double-blind reviews, editors with Rubriq accounts will have access to the identities of the reviewers on reports shared with them by authors. Permission to publish reviewer names alongside reviews will need to be up to each reviewer. We know that some journals are becoming more transparent with their peer review process in this way, so we are building in the ability for reviewers to give permission on a case-by-case basis to make the review and his or her identity public if the journal requests it.
Q: Can publishers use Rubriq in place of their own peer review processes?
A: Yes. We’ve talked with publishers that would like to use Rubriq directly for various reasons or recommend it to their authors. We don’t see Rubriq replacing peer review for most journals, but we hope that it can help provide editors with some initial insight and allow them to reduce their time to first decision. Some may use it as an initial filter by setting a threshold for a minimum score needed to submit.
We are finding that some publishers see value in suggesting Rubriq to rejected authors as a way to give the paper a second look without investing more resources, and at the same time, provide authors with useful guidance.