In today’s other post, Phil Davis takes a critical look at PubCred, a proposed solution to perceived problems with our current peer review system. Though it has some merit, the overall scheme is flawed, and in many ways resembles other recently proposed solutions for fixing what’s “wrong” with science. It creates a system that goes beyond merely fixing a flaw, a system that takes on a life of its own, becoming a career goal rather than an efficient means of getting researchers back on track.
Phil’s analysis of PubCred accurately points out the main flaw in the system — the notion that expertise is democratically distributed. It’s not. As an editor and as a reader, I want articles that have been reviewed by the best possible reviewers, the smartest, most relevant experts. And that often means that a small proportion of any given research community is going to perform more than their fair share of reviews. It’s not an ideal system, but it’s one that that’s better than the alternative, which allows unqualified reviewers more say and results in a lower quality of review and the publication of poorer papers.
PubCred would concentrate far too much power in the hands of the editors who choose peer reviewers. Arguments have recently been made, particularly in the peer review policy of PLoS ONE, that editors already hold too much power and that they shouldn’t be making value judgments about what should and shouldn’t be accepted for publication. The PubCred system takes this one step further, allowing editors to essentially decide who’s allowed to submit papers by strictly apportioning review assignments. An editor of the one journal in a small field could conceivably completely control progress and funding for that field.
It’s a system that penalizes productivity. It forces labs to limit the results they publish. One has to be stingy with one’s PubCreds. Perhaps some projects won’t get published in favor of spending one’s precious PubCreds on a different project that seems more high profile. This seems wrong in the era of abundance. It means burying research results, results that may in the long run prove more important than the ones that seemed bigger in the short term.
Another flaw is that the proposed system favors labs who spend lots of time peer reviewing rather than labs that spend more time doing research. And that’s where it starts to resemble proposed “karma” systems for measuring participation in Web 2.0 activities like blogging and post-publication peer review.
PubCred goes beyond merely rewarding those who review — it makes reviewing an absolute necessity, a requirement for having a career. It essentially starves out those who spend more time at the bench, and becomes an end unto itself. The biggest peer reviewer will have the strongest publication record. He’ll be able to use his extra PubCreds to get coauthorship on papers where he’s done nothing other than provide the PubCreds.
It’s yet another system designed to drive researchers away from the bench, to force them to spend their valuable time doing things other than experiments. While these proposals are all well-meaning, there’s an air of desperation hovering about them, an admittance that science is an increasingly difficult occupation. Success is based on achievement, on discovery of new knowledge. For those having trouble competing on that level, systems that offer similar career credit for other activities (where they excel) seem like an easy way around surmountable obstacles.
But participation in the community, either through peer review or commenting or whatever, is not as important as doing actual research. These sorts of things are peripheral activities, done in support of the main goal, not goals unto themselves. And PubCred is a system that makes research secondary to community participation. No reviews means no research output, and that’s putting the cart before the horse.
It’s unclear that any real crisis in peer review exists (the majority of researchers in this study felt otherwise), but even if one accepts that there is a crisis looming, the solution can’t lose sight of the actual goals of scientific research. We want our scientists to spend more time doing experiments, more time uncovering knowledge that will improve all of our lives. Being a scientist is already an extremely difficult way to make a living. Let’s find solutions that drive research forward instead of slowing it down. Credit should be awarded for peer review work, but it shouldn’t eclipse the credit given for actual research.