The second post in our series to celebrate Peer Review Week 2016 is by Tom Culley, a former policy advisor to the New Zealand Government and now the Marketing Director at Publons. Publons works with some of the world’s top publishers and journals to bring public recognition to the efforts of the expert peer reviewers and editors on the front lines of scholarly communication.

The theme for Peer Review Week 2016 is “Recognition for Review”, exploring all aspects of how all those participating in review activity should be recognized for their contributions. It’s heartening to see that researchers, publishers and institutions are increasingly starting to recognize the importance of peer review and the efforts of the experts offering their time to safeguard the quality and integrity of published research.

The level of interest around Peer Review Week, combined with the tens of thousands of researchers now showcasing their reviewing contributions to research via services like Publons’ and Elsevier’s recognition platforms, are cases-in-point that the industry is on the right track. There is still much more to do for reviewers. Funders need to recognize reviewing contributions and expertise. Training and mentorship for these guardians of research are also critical to improving review and research.

But what about the other experts involved in the scholarly publishing peer review process, namely, journal editors?  In these heady days of author and reviewer services, it’s tempting to picture a utopian future where papers are published online and review happens automatically, where the best research is easily discoverable and we surge toward the discoveries of tomorrow. The problem with this picture is that it ignores one critical node in the triangle of trustworthy and accurate research: the editor.

Editors play a central role in the research community, most of which happens behind the scenes. Editors manage and moderate over five million peer reviews a year; facilitate discussion around novel research; guide improvements to research; handle complex ethical issues; identify fabricated research; deal with fake reviewers. And they factor all of this into their publication decisions.

Are editors getting the recognition they deserve for their efforts? Are we doing enough to support editors and help them to improve? After all, in the current environment, they are the gatekeepers of a good chunk of the world’s research. If they do a poor job, the research community suffers, not to mention the journal that is relying on their expertise and judgement.

To understand the importance of a good editor to scholarly communication, it’s useful to imagine a world with a bad one.

The Bad

First, there is the impact of poor editorial work on the individual. Picture this: you’ve just spent the better part of a few years tirelessly conducting a novel piece of research and finally writing up a manuscript seeking publication. You’re excited to contribute to the sphere of human knowledge and move the needle in your field. You find a journal that’s fit for purpose and make your submission.

The handling editor then takes an eternity to commission unhelpful reviews from unsuitable reviewers. You’re asked to perform numerous revisions that are out of scope before finally getting a final reject decision. The whole process has taken a year, your paper is far worse as a result and you have to start over. This scenario is not uncommon and can potentially thwart one’s career.

What about the impact of poor editorial work on society? The rate of article retractions has been increasing in recent times. This isn’t always a bad thing. Some retractions are simply the scientific process in action, self-correction based on better information. Other retractions, however, are a result of false or misleading research slipping through the cracks, and when editors are ultimately responsible for either recommending or determining if a paper is published or not, they need to take some responsibility.

In a Guardian article in 2009, Ben Goldacre highlighted cases where editorial decisions led to, let’s say, ‘interesting’ papers making it through to publication in the journal Medical Hypotheses. One stood out as particularly reckless, where the authors, “devote [a whole page] to arguing that there cannot be lots of people dying of Aids in South Africa because the population of the country has grown in the past few years.” There’s more in this vein, but you get the point.

The editor of the journal at the time argued that provocative ideas should be free from censorship and that scientists should be able to pontificate in their internal literature. Unfortunately for the rest of us, the general public considers academic literature to be credible (probably due to an assumption of robust quality assurance processes provided by expert peer reviewers and academic editors).

The potentially harmful consequence is that bad ideas permeate throughout society and set back true advances in research and discovery. Science will self-correct in the long run, but that does little to prevent people with an appetite for immediate answers making bad decisions based on bad information in the short term.

The Good

There’s no point singling out any one brilliant piece of editorial work. We just need to look at the amazing catalogue of published research and truly incredible advances in human knowledge to date. Just as editors must take responsibility for their bad decisions, so they should also get credit for their positive achievements in scholarly communication.

Editors are essentially on the front lines – the sentinels of science, on the lookout for breakthrough papers that will advance our understanding, while warding off fabricated research and dealing with increasing instances of fake reviewers and ethical misconduct.

Editors deal with these challenges regularly and, for the most part, resolve them prior to publication. Let’s also not forget those instances where an editor’s expert advice and insight helps to improve the work of others and ultimately, the performance of papers’ and researchers’ careers.

How Are Editors Recognized Today?

For the most part, editorial work is done either for free, or for a very small stipend. If you’re on the board, you may get an annual trip to the board meeting, but most handling editors aren’t so lucky. It’s worth remembering, most handling editors are experts with their own research priorities. So why do they do it?

According to a survey of 400 handling editors that we carried out earlier this year, there are three main reasons:

1. For the most part, being asked to edit for a journal is considered good for a researcher’s career. They will be listed on the journal’s website as a contributing editor, providing external validation they’re considered an expert in their field.
2. To give back to the research community, much like peer review work.
3. In some cases, for a small amount of remuneration.[1]

A big problem with these incentives is that editors who go above and beyond get no more reward than those who just do the bare minimum. How can we better support and encourage all editors to do the best job they can? Are the efforts of academic editors getting the recognition and support they deserve? Don’t forget, these people have a direct impact on other researchers’ careers, the success of the journal and the flow of novel, trusted research to world.

What’s Missing? Can We Better Acknowledge and Support Editors?

With editors playing such a critical role, and the rate of retractions increasing, it’s worth asking if there is more we can do to support editors and improve editorial performance. Our survey provides some answers.

Public Recognition

Many in the industry assume editors are adequately acknowledged by being listed as an editorial board member on the journal’s website. Being listed as an editor undoubtedly has a positive impact on the career trajectory of many an expert but, as editors have repeatedly told us, it does little to convey the true nature or level of their editorial contributions.

89% of respondents to Publons’ survey said they wanted more formal, public and detailed recognition of their work. This is particularly important for academic editors — how do you distinguish between an editorial board member who handles 30 (or more) manuscripts a year and one who doesn’t handle any?

Facilitate Better Peer Review Processes

Helping editors find the best reviewers fast, every time, would drastically increase their ability to manage a robust and effective peer review process and arguably lead to better recommendations for publication.

Editors the world over all have the same answer to the question of the toughest part of their job: finding good reviewers who accept review invitations.  More support, tools, and training to help editors with finding motivated, expert reviewers would make a considerable difference.

Another major win would be to expand the pool of reviewers, most obviously by identifying (and providing peer review training for) early career researchers and researchers from non-Western countries.  Expanding the reviewer pool would spread out the peer review workload and give editors more options in finding motivated peer reviewers.

Performance Measures

Arguably what is really important is the “performance” of the manuscripts an editor handles. Editors want to be associated with top quality research publications. The existing key performance indicators for editorial performance center around journal impact factor, download figures and retraction numbers.

With the increasing popularity of altmetrics, however, the emphasis is increasingly on each individual manuscript. There is a real opportunity to build on existing metrics to provide editors with real-time measures about the performance and impact of the individual papers they handle over the short, medium and long term.

Better data on editorial decisions could then lead to greater guidance and information for editors to improve, as well as revealing the true nature of their impact on research, which in itself is a positive form of recognition.

So if it is true that more can be done for editors for the betterment of scholarly communication, what are we going to do about it? Publons has a number of plans in the pipeline, the first of which is to raise awareness of the importance of editors. To that end, platforms like Peer Review Week and the Scholarly Kitchen are a great place to start the conversation. Let’s make sure we keep it going.

[1] Publons internal Editor Survey 2016. Infographic available here.