A couple of weeks ago, I chaired a panel at the 2014 STM Spring Conference on Ethics and Trust in Journal Publishing: How Sound is the System? It’s an important topic, and one that I and all the panel members (John Bohannon – Science journalist, Phil Davis – consultant and fellow Scholarly Kitchen chef, Chris Graf – Wiley’s New Business Director, and Ivan Oransky – Retraction Watch co-editor) feel strongly about. At a time when more research articles are more readily available to more readers globally than ever before, it’s crucial we are confident that those papers meet the highest standards and, that on those occasions where they don’t, there is a sound system in place to revise or retract them.
So what can we do to make the publishing process more sound? This was one of the questions I posed to the panelists; their answers – a mix of the pragmatic and the aspirational – were thought-provoking and, in some cases, quite controversial!
Not surprisingly, many of the suggestions focused on peer review, including going all out for open peer review by deploying technology to facilitate that change – assuming that openness around the peer review process would facilitate scrutiny of that process, including accelerating the identification and correction of problems. A variation on this theme was the suggestion of making peer review part of the public record, along with the paper. While this wouldn’t necessarily be fully open in the sense that the reviewers themselves would be publicly identified, it would make the whole process more transparent. Although these suggestions represent significant changes to the peer review system, and would require a cultural and behavioral shift on the part of researchers and publishers alike, they are also plausible. Indeed, some organizations are already starting to experiment with these sorts of approaches – The EMBO Journal, for example, includes the following statement in its guidelines for authors:
The EMBO Journal makes the editorial process transparent for all accepted manuscripts, by publishing as an online supplementary document (the Peer Review Process File, PRPF) all correspondence between authors and the editorial office relevant to the decision process. This will include all referee comments directed to the authors, as well as the authors’ point-by-point responses. Internal communications and informal consultations between editors, editorial advisors or referees will remain excluded from these documents. Importantly, referee anonymity will be strictly maintained. Authors have the possibility to opt out of the transparent process at any stage prior to publication.
A much more radical suggestion, and one that would require far bigger cultural and behavioral changes, was to materially change the academic reward system, including an end to the ‘fetishization’ of the peer-reviewed paper as part of that system. A couple of years ago, fellow chef, David Smith, characterized the research process as follows:
- Scholar gets funding for research
- Scholar does research
- Scholar undertakes a process whereby they attempt to maximize the value of the research they’ve done by attempting to get as many papers out as possible, whilst simultaneously getting as much tenure/funding credit as possible for the same body of work (these things tend to trend against each other and you’ll note that there are two different definitions of value wrapped up there)
- Scholar selects journals in which to publish the work
- Publisher places successful works out for greater dissemination
- Fortune and glory follows (or not).
It’s fair to say that nothing much has changed since then (OA may now be mainstream, but it has had no real impact on this workflow) and, I suspect, nothing much is going to change any time soon. What’s more, it’s not an issue that we, as publishers, can (or should) influence; our mission is to serve the researchers and professionals working in our communities, not to dictate how or why they get hired or promoted.
However, it is within our control to implement one of the other radical suggestions from the panel – to create and implement a publication audit process for all journals. John and Ivan are both famous for ‘outing’ publishers whose publishing process is less than watertight – and in doing so, they are providing a valuable (if sometimes unpopular!) service. But, as they both noted, their work doesn’t constitute any kind of real (ie consistent, continuous) safety check. That would require publisher support and participation.
Ideally, an audit of this sort would be independent of the publishers themselves, but it is hard to see who – other than we – would be willing to pay for such a process. And publishers do have a good track record of collaborating to improve scholarly communications. Think CrossRef, ORCID, Research4Life, CLOCKSS, and more. In particular, many publishers (and the societies for which they publish) are members of COPE (the Committee on Publishing Ethics), which was formed in 1997 by a small group of medical journal editors to provide “advice to editors and publishers on all aspects of publication ethics and, in particular, how to handle cases of research and publication misconduct”.
At Wiley, we used the COPE toolkit to undertake our own ethics audit of our health science journals (covered by Chris at STM and at last year’s International Congress on Peer Review and Biomedical Publication), which we then used to help educate and encourage editors to improve their processes. Although this approach may not be scalable – and, in fact, the results were somewhat mixed – perhaps we could use elements of it to create a more efficient and effective audit system in future. It’s got to be better than waiting for John to sting us again, surely!?
With thanks to John Bohannon, Phil Davis, Chris Graf, and Ivan Oransky