(Please be aware, this article was posted on April 1st)

Starting today, anyone shopping on Amazon will soon be able to review manuscripts, just like pens, sneakers, and toiletry products.

Supporters of Amazon Peer Review™ believe that their simple 5-star rating system and unstructured comment boxes will greatly expand the community that participates in the review of scientific research.

Amazon Peer Review™ works by linking Amazon’s online store to bioRxiv, a rapidly expanding source for preprints in the biomedical sciences. Ratings will be tagged to manuscripts with an Amazon-branded quality badge.

1 star out of 5

Michael Eisen, UC Berkeley researcher and supporter of open peer review, is fully onboard with the Amazon-bioRxiv partnership, explaining his rationale on National Public Radio (NPR):

If you buy pen refills on Amazon, you get far more useful feedback about the benefits and deficits of a particular product than you do about a work of science that represents years and years of peoples’ work and millions of dollars of public investment

Responding to public comments at an industry-sponsored event, Jeff Bezos, CEO of Amazon, was thrilled that open online reviews were finally being embraced by scientists:

Why shouldn’t the public comment on a research paper on kidney disease? I have two kidneys, and so do most of my friends.

Critics, however, point out that many Amazon reviews are simply fake, written by automated software (bots) or sponsored by the company itself. Facebook admits that at least 12% of its accounts are fake, and between 9% and 15% of all active Twitter accounts are bots according to one recent study. Like Russia’s involvement in the last U.S. Presidential election, it may not be difficult to game Amazon Peer Review™ some worry. On the Internet, it’s getting much harder to detect if you’re a dog.

Defenders of open peer review systems argue that Artificial Intelligence (AI) will help scientists sniff out and expose fake preprint reviews. Indeed, AI software has already been implemented into peer review management systems to evaluate the future performance of papers although its efficacy has been disputed.

“AI sounds very sexy, like Scarlett Johansson in the film Her” tweeted Ross Mounce shortly after Amazon’s press release, adding “Researchers have many more pressing things to do with their time than reviewing papers.”

Phil Davis

Phil Davis

Phil Davis is a publishing consultant specializing in the statistical analysis of citation, readership, publication and survey data. He has a Ph.D. in science communication from Cornell University (2010), extensive experience as a science librarian (1995-2006) and was trained as a life scientist. https://phil-davis.org/

View All Posts by Phil Davis

Discussion

5 Thoughts on "Amazon Peer Review: Coming To A Preprint Near You"

“Alexa, round up the usual friendly suspects to rate, er, review, my latest anecdotal, er, observational study… Spot had a good review of my last one.”

I know, let’s call this new system “Autopilot”.

yes, but isn’t it a shame? In one case, it points out future possibilities by adding Watson’s off-spring to the review boards, given some of the problems faced with human labor, internationally. In the other case, it points to the fact that being published is, like getting a university degree, a signal to funding agencies and evaluators basing promotion and tenure by publication where the actual understanding of the value of the article that has been published and ranked even by an institution’s own faculty.

Except for a limited number of “medallion” journals, many struggle to meet publishers’ minimum requirements for volume and quality, requiring significant energy on the part of the editor and those who review materials due in part to the combination of the proliferation of journals and the increasing pressure, internationally, to publish. The ability to reproduce, even by the authors being a separate issue.

Leave a Comment