Waste Paper
Image by PhotoGraham via Flickr

Soon joining the ranks of many subscription-access publishers, the prestigious BMJ Group will launch their full open access journal, aptly named, BMJ Open. According to the job ad for the new managing editor:

BMJ Open is a pure Open Access journal – which will publish a high volume of medical research articles which might be rejected from the BMJ or BMJJs or which might be otherwise submitted to a journal outside the BMJ Group.

Now, there is nothing incorrect about this job ad.  BMJ clearly sees a market for their new open access journal.  But that’s it — a market, a business opportunity. The ad is devoid of any lofty goals and aspirations for how this journal is going to improve medicine.  BMJ Open represents a business decision to recapture manuscripts (and article processing fees) that would have been lost to other publishers.  It is a bulk publishing model, not unlike PLoS ONE.

Cascading Peer-Review

Focusing on reducing costs and improving efficiencies, many publishers have implemented services to redirect rejected manuscripts to related journals within their field.  Apart from reducing the redundancy of having a paper shepherded through the peer-review process a second or third time, internal manuscript and peer-review referral services offers real value to the submitting author — i.e., faster publishing.

For many authors resigned to the fact that they have been rejected by a top-tier journal, a second-tier specialist journal will do just fine; and barring those, a general archival journal may be better than no publication at all.

According to Matt Cockerill, Managing Director of BioMed Central, the future of BMC involves a model of cascading peer-review, where manuscripts rejected by premium titles (like Genome Biology), are transferred to moderate rejection-rate journals (BMC Bioinformatics, BMC Evolutionary Biology, and BMC Genomics), who, in turn, redirect rejected manuscripts down to BMC Research Notes, a journal of the broadest scope possible, whose article processing fees (US $940) are “deliberately kept low” to encourage publication:

BMC Research Notes is an open access journal publishing scientifically sound research across all fields of biology and medicine, enabling authors to publish updates to previous research, software tools and databases, data sets, small-scale clinical studies, and reports of confirmatory or ‘negative’ results. Additionally the journal welcomes descriptions of incremental improvements to methods as well as short correspondence items and hypotheses.

PloS Biology describes how it treats rejected articles in more ecumenical terms, allowing editors to suggest another PLoS journal that may be “more suitable” for publication and providing a mechanism for transferring the manuscript and reviews.

The Nature Publishing Group has implemented a similar transfer service so that authors can simply redirect rejected manuscripts from one journal to another, with Nature Communications residing at the end of the chain.

General open access journals may be perceived as a recycling bins for manuscripts that would likely be published elsewhere — mere business opportunities for publishers with a suite of established titles to recoup expenses that are invested every time an article is received, reviewed and then summarily rejected.  For publishers in the business of selling subscriptions, they can provide an additional revenue stream when authors, libraries, and funders are willing to pay the article processing charges and when there is no viable market for selling these articles to readers.

While a recycling bin may make economic sense for publisher interested in reducing transaction costs,  it does risk damaging — or at least diluting — the brand of the publisher, as Kent Anderson described in his recent post on the costs of rejection.

Earlier this year, the American Society for Microbiology, a publisher of 11 subscription-access journals, launched mBio with a different model in mind.  Unlike other publishers, the ASM does not view mBio as a trash bin for rejects. Rather, mBio‘s lofty goal is to publish “the best research in microbiology and allied fields” and will not accept submissions that were rejected by other ASM journals, although the converse is true — rejected mBio papers may be resubmitted to other ASM journals.

Taking pride at being at the bottom of the cascade, Rejecta Mathematica will only accept manuscripts that have been rejected by other math journals.  As a full open access journal, its business model is quite creative.  It does not charge article processing fees, but relies upon donations and the sales of t-shirts, coffee mugs and refrigerator magnets to cover expenses.  To date, it has published one issue containing just six articles.

Enhanced by Zemanta
Phil Davis

Phil Davis

Phil Davis is a publishing consultant specializing in the statistical analysis of citation, readership, publication and survey data. He has a Ph.D. in science communication from Cornell University (2010), extensive experience as a science librarian (1995-2006) and was trained as a life scientist. https://phil-davis.com/


31 Thoughts on "Cascading Peer-Review — The Future of Open Access?"

Given the tiered structure of the importance of research results, this cascading product line makes a great deal of sense for a large publisher, open access or otherwise. Journals aggregate then filter. There is no reason the filtering can’t be by level instead of pass-fail, if you own multiple levels. It saves everyone a lot of effort, which alone is enough to improve science.

Every research result can’t be top shelf in importance and this has nothing to do with quality. A lot of it is luck, just happening to ask the question that paid off. But any result might be useful, so should be published if possible.

I think there’s value in these sorts of journals. There’s a lot of research done that perhaps doesn’t tell a complete or dazzling story, but that shouldn’t be lost, and it’s useful to have a place where it is made available (and searchable).

As an example, a very prominent lab I know published a paper in one of these low-end journals after rejection from two higher-rated journals. The reviews asked for additional sets of experiments to complete the story. The graduate student who had done the work had graduated and moved on to a new lab, and no one was going to pick up the project. So the choice was either to bury the data where it would never be seen, or to at least get some minor credit for it, and make it available in case it was of use to anyone else.

So these sorts of journals play a valuable role. They’re not the kind of landing place you initially aim for when doing research, but do provide a needed outlet. The question I have is how many the market will bear. There’s a frictionless path one can take by staying in the same publishing house as a paper cascades down. But does every publisher need to create one of these outlets? And if you make the effort to move the rejected paper from one publishing house to the next, why not aim for a better journal rather than accepting one’s fate and just dumping the paper off in a reject repository?

One wonders, as these lower-quality journals proliferate, how they will be distinguished from those journals that claim to be peer reviewed and purport to have editorial boards and all the regular academic paraphernalia, but are in fact simply scam operations? It has become so easy to set up OA journals that the world is going to be presented with the additional task of distinguishing the bona fide journals from the scam operations.

I had not heard of the “scam journal” problem. Anything written on this? How do they operate? Are they defrauding the researchers whose work they publish, or creating fake work? If the latter how do they make money?

Lower tier journals published by known publishers would have name brand recognition. But I still object to calling it lower quality. It is mostly just lower importance.

British journalist Richard Poynder, who has a blog, has written the most about scam OA publishers. (I will not name names here as some of these publishers are very litigious.) Their modus operandi is to announce a new journal that purportedly has a distinguished editorial board and then to invite submissions, which are then accepted after presumed peer review, leading to a charge for the author to have them published. One of the publishers I’ve investigated will not tell you who the members of the editorial board are, but simply provides a listing of scholars by subject area who presumably are called up to do reviews. When I checked with one of the people whose names I knew, he had never heard of the journal or the publisher, and indeed the area he was listed as being an expert in was not his field at all!

P.S. I accept David’s correction that these newer journals are publishing articles of lesser importance rather than, necessarily, lower quality.

May I add a few clarifications to your comment in your post about the Nature journal system? It was launched in 2005, so predates these services and journals you discuss here. See http://www.nature.com/authors/author_services/transfer_manuscripts.html for a description. Specifically, it is a service in which the author may resubmit a rejected manuscript from one journal to another, without having to fill out the submission forms again. If the transfer is between journals in the Nature family, the receiving journal’s editors can see the identity of the referees and may decide not to peer-review further. These journals, however, are all very high-quality journals in their disciplines (by independent measures, ie by measures not by their publisher). Each of these journals is discipline-specific, and has an extremely high rate of rejection. Nature Communications launched earlier this year and has been added to the service – we don’t yet know about the pattern of submissions as insufficient time has elapsed, but we do anticipate that papers in disciplines not covered by any of the 14 monthly research journals will be stronly evident if not predominant.

NPG also publishes quite a few much more specialist journals, that are not “Nature” journals – most are published on behalf of scientific societies. These journals are also in the manuscript transfer system, if a rejected author wishes he/she can transfer submission to one of these journals, but the identity of the referees is not transferred. This service is an author service, ie it saves the author from having to re-enter their details in the ms submission website, in the main.

I think this process (as detailed in the link I’ve provided) is rather different from the “cascading” model you discuss in your post.

First, thank you for the details of the Nature manuscript transfer service.

“Transfer” as it is used here, is a neutral term and implies that a rejected manuscripts could — theoretically — move to a more prestigious journal or a journal of similar credibility.

Indeed, PLoS talks of redirecting manuscripts to “more suitable” outlets seeming to imply that an article rejected from PLoS ONE have an equal chance of being accepted in PLoS Biology as the reverse situation.

Yet, studies of the trajectory of rejected articles demonstrate, with few exceptions, that manuscripts move downward through the perceived pecking-order of journals, from higher-prestigious journals to lower prestigious titles.

In this sense, the use of the term “cascade” is not misused here but reflects the natural flow of most rejected manuscripts. Matt Cockerill of BioMed Central uses the term “cascade” and has direction arrows in his presentations diagramming how manuscripts are redirected downward through BMC’s hierarchy of journals.

I realize that “cascade” may be taken as a derogatory term, although it does reflect the fate of most rejected manuscripts.

The so-called trajectory of rejection is a fascinating topic in the science of science. Can you point us to these studies?

My concern is that there may be two statistical dimensions being squashed into one, namely importance and narrowness. Narrow journals tend to have less impact than broader journals, the extreme cases being Science and Nature, which cover everything.

What is published in a narrow journal may be extremely important to the narrow audience, while of little importance to the broader one. Yet the work of science tends to be done in the narrow space, like mining.

Here are a number of the studies that track the trajectory of rejection. Each study is only able to capture one step in the peer review cascade:

Wijnhoven, B. P. L., & Dejong, C. H. C. 2010. Fate of manuscripts declined by the British Journal of Surgery. British Journal of Surgery 97: 450-454. http://dx.doi.org/10.1002/bjs.6880

McDonald, R. J., Cloft, H. J., & Kallmes, D. F. 2009. Fate of Manuscripts Previously Rejected by the American Journal of Neuroradiology: A Follow-Up Analysis. American Journal of Neuroradiology 30: 253-256. http://dx.doi.org/10.3174/ajnr.A1366

Groves, T. 2009. Nine in 10 articles rejected by NEJM appear in another journal. BMJ 339: b3777. http://dx.doi.org/10.1136/bmj.b3777

Hall, S. A., & Wilcox, A. J. 2007. The Fate of Epidemiologic Manuscripts: A Study of Papers Submitted to Epidemiology. Epidemiology 18: 262-265. http://dx.doi.org/10.1097/01.ede.0000254668.63378.32

Liesegang, T. J., Shaikh, M., & Crook, J. E. 2007. The outcome of manuscripts submitted to the American Journal of Ophthalmology between 2002 and 2003. American Journal of Ophthalmology 143: 551-560. http://dx.doi.org/10.1016/j.ajo.2006.12.004

Opthof, T., Furstner, F., van Geer, M., & Coronel, R. 2000. Regrets or no regrets? No regrets! The fate of rejected manuscripts. Cardiovascular Research 45: 255-258. http://dx.doi.org/10.1016/s0008-6363(99)00339-9

Cronin, B., & McKenzie, G. 1992. Documentation note: the trajectory of rejection. Journal of Documentation 48: 310-317.

Two more references on the fate of rejected manuscripts, both confirming the other studies that most manuscripts are eventually published in less prestigious outlets:

Ray, J., Berkwits, M., & Davidoff, F. (2000). The fate of manuscripts rejected by a general medical journal. The American Journal of Medicine, 109(2), 131-135. http://dx.doi.org/10.1016/S0002-9343(00)00450-2

Chew, F. S. (1991). Fate of manuscripts rejected for publication in the AJR. Am. J. Roentgenol., 156(3), 627-632.

Thanks Phil. The first interesting question is how is “prestigious” being defined in these studies? Can you say or do I have to (gasp, choke) read them? Perhaps you can respond with a new thread below, so I can get out of these narrow confines. I am thinking that mapping the cascade might be very useful, not only for understanding science but also for reducing the burden or re-submission. Or so I submit.

Dear Phil

Firstly, thanks for not circulating this job ad until after someone was appointed. (Competing interest: I was the successful candidate.)

You don’t seem to come down on a position about cascading peer review or the journals you refer to, but you seem concerned at least by the launch of BMJ Open. In which case, it’s arguably unfair to judge it solely on the job ad; you could at least have quoted the journal’s aims and scope or FAQs, which are available on the BMJ Open blog you link to. They were both published well in advance of your post above. Section 1 of the FAQs states: ‘Transparency and openness define BMJ Open’ before explaining how. Is that sufficiently ‘lofty’? Unless your point is simply that BMJ Group needs to spice up its job ads?

BMJ Open is using open peer review (reviewers and authors will be made known to each other and reviews will be published with previous versions as prepublication history). We are also looking at ways in which we can encourage and facilitate data sharing and use of repositories. BMJ Open’s mark will most likely be made by being in the vanguard of the group’s attempts to bring greater transparency to the publishing process, over and above through publishing well-conducted research for reasons some of which are identified by the comments above.

Lastly, I’m sure many who work in publishing would be interested to know which of the tasks (as described in the advert) you consider menial and/or low status: “ensur[ing] that the processes for submission, peer-review and acceptance of content are effective and efficient and are in accordance with business expectations and the allocated budget. … responsibility for ensuring the integrity and quality control of the scientific content published by BMJ Open and for ensuring the implementation of and adherence to BMJ editorial policies”.

Whereas every editor should attempt to ensure the highest level of quality and integrity to the editorial process — and I’m certain that you will do the same with BMJ Open — this ad undersells the real value of your journal to medicine by focusing on the high volume of rejected manuscripts aspects.

As I wrote in the post, PLoS is decidedly neutral in how they describe PLoS ONE, and BioMed Central seems to embrace the notion of cascading peer review.

Marketing creates a perception, whether that perception is accurate or distorted. If I have any view on BMJ Open, it is that your employer could have done a better job marketing your position.

David Wojick:
…how is “prestigious” being defined in these studies? … I am thinking that mapping the cascade might be very useful, not only for understanding science but also for reducing the burden or re-submission. Or so I submit.

In the 9 studies I cited (here and here), the authors operationalize the prestige of a journal as its Impact Factor (one also uses Immediacy Index). In a few cases, the authors also classify whether the target journal is a general field journal or a specialist journal.

Because there is no single review software operating on a single database, it would be difficult to accurately map the path of rejected manuscripts. I imagine that asking authors to recount the path of their last submission may allow a researcher to build such a map, but it would require asking a lot of authors.

Note also that these studies appear to cover cascades in medical journals, so it’s unclear how relevant the findings are to more general science journals.

It should not be assumed that all papers published in open access journals have been rejected from other journals. I have just had a paper accepted by BMJ Open. This paper was not submitted elsewhere: BMJ Open was my first choice. Open access journals are an appropriate forum for methodological studies such as mine. This is the 3rd time I have published in an OA journal. The first two in BMC Medical Research Methodology, and neither were submitted elsewhere. All 3 papers were thoroughly peer reviewed and improved considerably after review.
I fully support the open-access model as does the Wellcome Trust who currently fund my work.

No one is assuming that open access journals publish only rejected articles, although many publishers clearly see their OA titles as fulfilling the role of recapturing articles that might be resubmitted and published elsewhere because they were rejected by other journals within their stable of control.

I have to say that when I am considering where to publish, I prefer OA journals because I am most interested in the widest dissemination possible. I publish more in Against the Grain than anywhere else because ATG allows OA two months after publication. But then I’m not a scholar worrying about publishing in the most “prestigious” journals with the highest “impact factor” to help advance in my career!

I agree with Sandy. However, I believe that publishing in an OA journal has helped advance my career and I have been pleasantly surprised by the number of citations for my two previous OA publications.

Comments are closed.