Business has only two functions – marketing and innovation. — Milan Kundera
The fundamental question dogging the concept of post-publication peer-review as imagined by alt-metrics and open access (OA) advocates stems from this Milan Kundera quote — that is, is it an idea being marketed or a form of innovation?
To be a form of innovation, it would need to be superior in some way — more expeditious, more reliable, more likely to be used, or more convenient. If it isn’t any of these things, then it may be more heat than light.
A recent column in Wired by Dan Cohen argues that OA can’t be effective if post-publication peer-review isn’t made as robust as possible, and seems to dismiss pre-publcation peer-review as undesirable because it is “closed.” He argues that creating a system of awards — as opposed to rewards — is one possible way to do this.
We need a sensible shift towards an acceptable form of post-publication, rather than traditional pre-publication peer review. . . . Yet we already have an obvious form of post-publication peer review in wide use: awards.
Cohen and many others fail to appreciate that pre-publication peer-review doesn’t preclude post-publication peer-review. Everyone acknowledges that filters are important to differentiate quality and relevance, but the pressure to choose between pre-publication peer-review and post-publication peer-review is a false choice. The two are not mutually exclusive.
Nor is pre-publication peer-review anything more than a tool, one of many tools an editor or a journal can put against content to make it more likely to bring forward high-quality content that is relevant to its audience. Science is a process which includes publication, and there is no magical step that ensures perfection. Peer-review is part of the process, not the entire solution.
So to argue that you need to replace pre-publication peer-review with post-publication peer-review doesn’t make sense. OA needs pre-publication peer-review as much as any other business model around scientific content. Cohen realizes that validation is key to acceptance, but believes that awards can provide validation:
Without this validation, an open access system can never work on the demand side.
The distinction between reward and awards turns out to be a crucial one which Cohen misses. And that’s unfortunate. Because if he had addressed it, the entire premise of his essay would have changed.
Awards are usually meaningless in the long run. Win Teach of the Year? Nice, but if you’re teaching a course that’s doomed, it probably won’t save your job. Win an Oscar? Commendable, but it won’t make you into a box office champ like Samuel L. Jackson. Win a Grammy? Sure, it can help if you also have commercial appeal, but many Grammy winners fade into oblivion.
Rewards, on the other hand, combine both power and viability, and therefore beat awards hands down. Scientists seek grants, tenure advancement, higher positions in the academic hierarchy — in other words, the rewards of academic life, which emanate from other rewards like published studies, successful research grants, and robust and productive labs.
Rewards incentivize science, which is the ultimate form of post-publication peer-review. Science is, in fact, completely predicated on the notion of post-publication peer-review. The two are synonymous. Scientist A publishes his results. Scientist B reviews those post-publication, and sees a flaw. Scientist B can’t reproduce Scientist A’s results, but gets more interesting, different, and better results. Scientist B publishes her results, referring to Scientist A’s paper, and everyone following the field realizes what has occurred. And Scientist B gets some nice rewards — a better publication, more citations, wide recognition, and likely some leverage in grant applications and within her institution.
Some people continue to fail to appreciate what exists and seek to invent something that is less practical and less likely to succeed. While we need to test our system of rewards to ensure they align with what we want to achieve, a system of awards to divert scientists’ attention into a weak form of post-publication chatter and out of a powerful, rewards-based system of science seems unwise.