Editor’s Note:  Today’s post is by Seth Denbo, Director of Scholarly Communication and Digital Initiatives at the American Historical Association.

Open peer review hasn’t caught on in the humanities. Nearly ten years ago, a few notable experiments attracted the attention of the New York Times. The “Web Alternative to the Venerable Peer Review,” as the headline in the print edition on August 24, 2010 dubbed it, was presented as an innovation that would revolutionize the way scholars evaluated each other’s work. Breathlessly excited about the potential of web-based open review for “generating discussion, improving works in progress, and sharing information rapidly,” the Times contrasted this with what was presented as the purely “up-or-down judgment” of customary review practices. Openness was said to be central to the attractiveness of these new forms of peer review.

Flash forward to the present, and little widespread change in humanities peer review has occurred. Many articles on peer review have pointed out that the systematic practices we think of as central to scholarship and scholarly communication evolved as recently as the mid-20th century. Melinda Baldwin has written on how peer review did not come to be seen as necessary for scholarly legitimacy until the Cold War. Ben Schmidt has shown that the phrase “peer review” doesn’t enter the lexicon until the 1970s. Despite the relative recent emergence of systematic practices, peer review is central to scholarship. And, within a range of different ways of organizing review and masking the identity of author, reviewer, or both, a more-or-less closed process still dominates in humanities journals and book publishing. These long-standing practices still seem to provide editors with the evaluation they require to maintain quality and the feedback that assists authors in improving their work. Alex Lichtenstein, editor of the American Historical Review (AHR), recently wrote “as an editor I especially value the developmental as well as evaluative role” provided by the current double-blind peer review practices and structures that he directs as editor.

Despite his commitment to double-blind review, Lichtenstein is overseeing the AHR’s first foray into experimenting with open review. “History Can be Open Source: Democratic Dreams and the Rise of Digital History” by Joseph L. Locke and Ben Wright is currently posted on ahropenreview.com for an open, public comment period that will run until early April. In parallel, the editors have invited several reviewers to submit more traditional peer reports. Those reviewers have been given the option of anonymity, but their reviews will be public.

Peer review practices vary between the sciences and the humanities, and even by discipline within those domains. Fittingly, given the relative preeminence of books in many humanities disciplines, where reputations and careers hang much more on monographs than on journal articles (which are often seen as a step on the way to the book), the earliest humanities experiments in open review were book manuscripts. Kathleen Fitzpatrick, now Director of Digital Humanities and Professor of English at Michigan State University, posted the manuscript of her book Planned Obsolescence: Publishing, Technology, and the Future of the Academy online for open comment in 2009, modeling new approaches to digital scholarly communication explored in the book.

Experiments of this kind have often been conducted by academics like Fitzpatrick with a scholarly interest in new media or digital humanities. When the prestigious Shakespeare Quarterly trialed open review in 2010, it was for a special issue on the Bard and new media edited by Katherine Rowe, a leading digital humanist. While the organizers and participants generally saw the process as beneficial, it has not been repeated. In 2011, Jack Dougherty and Kristin Nawrotski edited Writing History in the Digital Age — a book about how the Internet has changed the way historians teach, write, research, and publish — and the editors invited contributors to share their drafts for open comment on the web. The German digital humanities journal ZfdG, one of the few with a standard open review policy, allows authors an interesting choice. Contributors can opt for either a double-blind evaluation process before their article is published, or open, post-publication review with subsequent opportunity to revise the article. The Programming Historian, a highly innovative publication that provides online tutorials on digital tools and methods for historical research, also uses open review as part of their regular editorial practice. Palgrave Macmillan’s medieval cultural studies journal postmedieval is one of very few humanities journals that has used open review more consistently. Reviews for several special issues of the journal tackling a diversity of topics have been subject to what they call crowd review experiments since 2011.

Journals in the sciences, by contrast, have moved much deeper into the use of open online review. When Ann Michael asked several of the Scholarly Kitchen Chefs about the future of peer review for a 2017 post, Michael Clarke described a state of “fecund experimentation and roiling debate,” including journals exploring various forms and levels of openness. Some scientific journals use open reviews of experimental design to try to improve methodologies and research quality before any experiments are conducted. While there is little agreement about what open peer review is and almost as many different practices as there are journals doing it — a 2017 F1000 Research article identified over 120 different definitions — it has become an important enough part of the landscape that it has generated its own studies. A number of high prestige medical journals, such as the British Journal of Surgery, BMJ, and JAMA, have conducted studies into the effectiveness and quality of various forms of open peer review. But defining quality is a complex and often subjective process.

Why, then, have humanities journals and scholars not taken up open review practices in more than a few notable instances? Openness as both value and practice has infused the discourse around scholarship and the communication of ideas among many in the humanities, but old practices die hard. Is it that the current models meet the needs of these disciplines? Or is it merely a resistance to change? One of the primary complaints about peer review among scientists is the lack of consistency. Most humanities disciplines are comfortable with, and even rely upon, certain kinds of subjective judgments by scholars. It could be that with this outlook, authors and editors in the humanities feel less keenly the necessity to reform review to prevent inconsistency between reviewers.

And there is some evidence that there are benefits to double-blind review that help with issues of equity and inclusion. A 2017 study published by Critical Inquiry suggests that journals that maintain double-blind review practices have a broader base of authors (at least as defined by the institution at which the authors received their PhD and where they are employed). Another study in computer science showed that single-blind reviewing confers advantages to well-known authors and those from prestigious institutions. Researchers have studied and documented the extent of gender bias when peer review is not double blind, showing that shown that papers authored by women are accepted at lower rates than those by men. The evidence should make editors and scholars pause and consider the benefits of blind review.

Good peer review has never just been gatekeeping. In the humanities, reader’s reports are often longer than this blog post. Reviewers engage with the arguments, refer the author to other literature and sources they may have missed, and provide developmental editing suggestions. This leads to better research and better publications. There’s no reason why this can’t happen in an open environment, and in the case of many of the experiments listed above, it has. But any innovation should ensure that we preserve the good of the way things have been done, and take advantage of new technologies, practices, and processes in ways that promote excellence in scholarship.