In September 2014, a group of more than 40 Australian journal editors submitted a letter entitled Journal Reviewing and Editing: Institutional Support is Essential to the Australian Research Council, the National Health & Medical Research Council of Australia, Universities Australia, the Australian academies, and all Australian Deputy Vice Chancellors of Research. In it, they argued that the review and editing of scholarly papers is a critical element of academics’ work and should be recognized as such by their institutions and funding bodies.
A couple of years before this, Sense About Science’s early career researcher group, Voice of Young Science (VOYS), wrote an open letter to the then CEO of the Higher Education Funding Council for England, Sir Alan Langland, with a similar request – that the organization recognize the contribution that researchers make to science through the peer review of articles and grant applications.
Other than this commentary (by one of the signatories) both organizations have, to date, offered nothing more than brief acknowledgements.
Which is too bad because, despite recent criticism in some media (see, for example, this Slate post), peer review remains central to scholarly communication. In fact, there is evidence to suggest that researchers, are increasingly satisfied with peer review (69% in Sense About Science’s 2009 survey on peer review, compared with 64% in 2007), that they believe it is trustworthy (the 2013 CIBER/University of Tennessee study on trust in scholarly communications states that “researchers agreed that peer reviewed journals were the most trusted information source of all”), and that it improves their research (91% according to Sense About Science again).
However, it’s hard to argue with the perception that peer review is facing, if not a crisis then at least challenging times. With research output – and articles – continuing to grow at 3-4% annually, more reviewers than ever are needed.
The Australian letter highlights the concerns of today’s journal editors, that, “To help maintain the publication quality that Universities and the ARC expect and rely upon, research-active academic staff must be involved [in] peer-review or editorial activities.” While the VOYS letter reflects the concerns of the next generation of reviewers and editors, who believe that,“ Without recognition in the REF we risk reviewing of both papers and grant applications becoming a marginal activity and inevitably inconsistent and shoddy.”
Specific issues include:
- The tension between the pressure for researchers to publish in peer-reviewed journals and the lack of support for peer review from their institutions and funders. Perhaps my favorite quote from the Australian letter is: “no Nobel Prize was won by an unpublished research work and the peers who reviewed, published and judged the work had to be people whose opinions mattered within the discipline.” Although peer review is rarely rewarded in monetary terms, researchers have historically been happy to participate in the process, mainly for altruistic reasons. For example, the 2009 Sense About Science survey found that 90% of respondents did so because they believe they are playing an active role in the community; only 16% said that increasing their chances of having future papers accepted is a reason to review. However, many institutions and funders fail to recognize or reward their researchers for the time they spend on peer review – either as reviewers themselves or training others.
- The excessive value placed on publication output by institutions and funders, as opposed to other contributions researchers make to the scholarly endeavor. As the signatories of the Australian letter point out, “The ERA procedures effectively mean that certain research activities are rewarded while other academic activities are not; and that universities suffer financial consequences if their academic staff do not privilege [sic] the winning of large grants and publication of articles in prestigious, high quality journals over all other work.” But without the work of editors and peer reviewers (not to mention the many other contributors), research papers would never see the light of day. Ironically, this is especially true of articles published in journals with high rejection rates – which also typically happen to be exactly those aforementioned “prestigious, high quality journals”.
- The increasing difficulty of finding researchers who are willing to undertake peer review. With research output continuing to increase at 3-4% annually, finding enough knowledgeable, well-trained reviewers is a challenge. VOYS suggests that reviewing should be “approached professionally and seriously, enabling senior researchers to spend time mentoring early career researchers like ourselves in these activities.” But without support from their institutions and funders, it’s increasingly difficult for senior researchers to prioritize training the next generation of peer reviewers over their own need to publish or perish.
- The particular challenge of finding reviewers for special issues which, the Australian letter suggests, are increasing in frequency. According to the signatories, “For academics the publication of a Special Issue on a particular theme – especially if it reflects research output for an ARC grant, is deemed an achievement — while for editors the escalating difficulty of finding one reader who is prepared to evaluate the collection as a whole to see that it has integrity and thematic coherence is ignored.”
So, what to do? Both groups essentially make the same request, summed up in the Australian letter as follows:
First, there should be much more explicit requirement and recognition within Universities of the professional service requirements of academics. All academics engaged in publishing should also be involved in reviewing or similar activities, and Universities requiring staff to meet publication targets should also be setting professional service targets. Second, Universities need incentives to develop and maintain these professional service requirements. Inclusion of professional services to journals within the ERA framework is a key option to achieve this.
These modest requests, if implemented widely by institutions and funders, would have an immediate and positive impact on scholarly communications, especially in countries with a centralized approach to the evaluation of institutions for funding purposes, such as Australia and the UK.
Publishers – and societies – are well placed to help maximize this impact. For example, formal peer review training is something that many of our organizations are already investing in individually and would, I believe, be happy to support more rigorously, perhaps through our industry organizations, such as SSP. In particular, perhaps we should be looking to support peer review training for Chinese scholars, since much of the increase is in submissions is coming from China. Continuing experimentation with different peer review models is also critical – again, something that publishers are taking the lead on. This includes portable peer review (see here and here, for example); new services such as PRE, which was recently acquired by STRIATUS/JBJS, publisher of the Journal of Bone & Joint Surgery; and post-publication review, for example, at F1000. Initiatives like CREdiT, which proposes a contributor taxonomy as a way of facilitating recognition for all contributors to a research paper, not just the author(s), also have a role to play.
Then, of course, there’s also the thorny question of whether researchers should be paid to do peer review – and some certainly feel strongly that they should. The same Sense About Science survey found that reviewers were split on this topic – slightly more than half felt that some kind of payment in kind (such as a free subscription) would be appropriate, just over 40% wanted to be paid (though unsurprisingly this dropped to 6.5% if authors had to pay for it themselves), and 40% would be satisfied simply with some form of acknowledgement in the journal.
So, with somewhat belated thanks to VOYS and the group of Australian editors, here’s to what could be the start of a good, positive debate about how to ensure the essential contribution that peer review and peer reviewers play in scholarly communication is fully recognized, rewarded, and supported – by funders and institutions, promotion and hiring committees, publishers and societies.