Editor’s Note: Welcome to the dog days of summer. So many of the pressing issues in scholarly communications are not new, but rather continuations that have evolved over years if not decades. As our Chefs enjoy the waning days of summer, we’re taking a look back at earlier posts on subjects and offering thoughts on what, if anything, has changed.
Last week’s post by Alice Meadows showcased some of the impressive new technologies that are better enabling us to track the work done by researchers, in particular, work that has previously remained somewhat invisible. But even with these abilities in-hand, functional questions still remain about who would grant that credit and what form it would take.
I was inspired by the comments on Alice’s post to go back and look at The Scholarly Kitchen archives to see how long the conversation over credit for peer review has been going on. The first mention I can find is in this 2009 piece by Phil Davis, looking at a survey done that year by Sense About Science. One of the key takeaways from that survey is that acknowledgement, either in the form of payment or career credit was sought after by more than half of the respondents. So although the conversation probably goes back further, we can definitively say that this has been in discussion for more than ten years now. Systems offering credit for peer review date back to at least 2010, with the proposal for PubCreds, an attempt at privatizing the peer review commons that never got off the ground.
So I’ll pose the question as we revisit my 2015 post about credit for peer review — are we any closer to realizing it?
Offering career credit to researchers for performing peer review seems like a no-brainer, right? Peer review is essential for our system of research, and study after study confirms that researchers consider it tremendously important. Funding agencies and journal publishers alike rely on researchers to provide rigorous review to aid in making decisions about who to fund and which papers to publish. On the surface it would seem to make sense to formalize this activity as a part of the career responsibilities of an academic researcher. But as one delves into the specifics of creating such a system, some major roadblocks arise.
One such problem falls into the realm of volunteerism and motivation. Right now, most academics see performing peer review as a service to the community. It’s important to the advancement of the field and so they volunteer their time. If instead we turn peer review into a mandatory career requirement that is rewarded with credit, it changes the nature of the behavior. If we set standards (you must do X peer reviews per year) people will then work to those standards rather than the more generous acts we see today, where good samaritans (and good reviewers) take on much larger workloads.
Economists suggest that incentives (a form of reward) changes motivation, some of which will be actualized by real behavioral change. Educator Alfie Kohn talks about how behaviors change in light of offering rewards in one of his books on parenting:
…there are actually different kinds of motivation. Most psychologists distinguish between the intrinsic kind and the extrinsic kind. Intrinsic motivation basically means you like what you’re doing for its own sake, whereas extrinsic motivation means you do something as a means to an end — in order to get a reward or avoid a punishment…extrinsic motivation is likely to erode intrinsic motivation…The more that people are rewarded for doing something, the more likely they are to lose interest in whatever they had to do to get the reward.
Plugging peer review into a rote system of requirements may threaten both the participation levels and the rigor and enthusiasm with which many approach the task.
A second problem with peer review credit schemes (and perhaps I’m arguing against my own interests here) is the increased power it places in the hands of publishers and editors. Researchers have little control over whether they are asked to do peer reviews. If your career is dependent upon getting a journal to ask you to review a paper, what happens if you’re not on their list? An early career researcher who is not well-known in their field is at an automatic career disadvantage under such a scheme. Many are already resentful of the “kingmaking” ability of editors of journals like Science, Nature and Cell. Turning over even more power over academic career success to publishers might not be so well-received.
Perhaps the biggest problem of all comes when we ask the simple question that must always be asked when new changes to the academic career structure or the scholarly publishing ecosystem are proposed: Who cares? Not “who cares” as in “peer review is unimportant and no one should care about it”, but “who cares” as in “who exactly are we asking to grant credit here?”
As we are constantly reminded, the two things that matter most to academic researchers are career advancement and funding (and the more cynical among us suspect that the former is primarily dependent on one’s ability to secure the latter).
If I was an administrator at a research institution, I’m not sure I’d want my researchers spending an enormous amount of their time helping to improve the papers of researchers at other institutions. If I was a particlarly wise administrator able to see the big picture, I would understand the value of peer review and how it is necessary for the advancement of knowledge. So I’d know some amount of credit is due. But it’s not the primary reason I hired those researchers, nor is it something I want them spending a lot of their time doing. Their job is to do original research.
Similarly, a funding agency gives a researcher a grant to do research. A diabetes foundation is looking to fund research to cure the disease and likely wants fundees spending their time doing original research, not reviewing papers from other researchers. How much should they reward fundees for doing something other than what they’ve been funded to do? And back at that research institution, if much of the tenure decision (at least in the sciences) is based on how much funding one can bring in, then if peer review doesn’t bring in funding, it won’t matter all that much.
How much career credit should a researcher really expect to get for performing peer review? I suspect that at best it will be a few small percent of the overall picture, more likely a box on a checklist–did you do any peer review? If yes, then you get a small bonus amount of credit. No one is going to get hired, tenure or funding based on a stellar record of peer reviewing lots and lots of papers. There’s a different job where you get rewarded for that — it’s called “editor”.
The proposed peer review credit systems currently under examination, both commercial and community-based seem like overkill. Many systems offer extensive tracking, point systems and review of reviewers which may be unnecessarily complex for a yes/no question. As Joe Esposito has trained us to ask, this is at best a feature, certainly not a product nor a business.
Perhaps something along the lines of the work ORCID and CASRAI are doing will suffice in the end. Tag the activity to the researcher’s identifier and offer a simple yes/no or a tally of peer review events for the year. Do we really need anything more than that?