Editor’s Note: Today’s post is by Victoria Corless. Victoria is associate director of community development at Taylor & Francis.

Much of the debate about AI in peer review is centred around whether AI systems are accurate enough to perform tasks currently undertaken by editors and reviewers. Generative AI can already help summarize manuscripts, check references, and even flag methodological problems, while other AI-based tools can help match papers to reviewers. Currently, the use of AI for these tasks varies by publisher and most are taking a cautious approach. But what will happen to the parts of peer review that are harder to define, let alone automate?

While there are both supporters and detractors, the emerging consensus is that AI should assist rather than replace human judgement, leaving human expertise and critical thinking as essential elements in the peer review process. However, what appears to be missing in the discourse is a more difficult question: What, exactly, are the human parts of peer review doing, and how might AI shape the conditions in which that judgement takes place?

This became apparent in a series of interviews conducted with researchers and Taylor & Francis journal editors in Germany, across disciplines. The interviews weren’t designed as formal qualitative research; they were semi-structured conversations for a publisher-run series. But across the conversations, some participants described forms of judgement that depended on highly specialized knowledge that is difficult to reduce to explicit criteria.

Part of the answer may lie in a form of expert knowledge that is difficult even for experts to articulate. This is tacit knowledge, and it is knowledge that experts demonstrate in practice without being able to fully specify the rules by which they do so. This does not necessarily mean an inexplicable “gut feeling.” Rather, expert judgement can be the product of years of accumulated experience: knowing which results are unusual but plausible, which inconsistencies are meaningful, which omissions matter, and which apparently minor details should trigger further scrutiny. When it comes to reviewing papers, much of that knowledge may be difficult to translate into a set of explicit rules that AI can follow because experts themselves may have acquired it gradually.

Wilhelm Windisch, a professor at Technical University of Munich and editor-in-chief of Archives of Animal Nutrition explained that some of the manuscripts he comes across contain claims that only an experienced specialist could recognize as impossible. “In some highly specialized subfields, such as my own, it may be the case where there are only 20–30 people worldwide who might be genuinely qualified to review a particular type of manuscript.”

He offered an example from his own field of animal science, where an experienced researcher who has conducted livestock studies may recognize that a reported performance result is simply physiologically implausible or a calculated value that could not make sense.

Other problems are harder to spot when generative AI is involved as it can produce fluent prose and attach apparently relevant citations to a claim, while an experienced reviewer may notice that the cited literature does not actually support the statement being made. Identifying that mismatch, Windisch argues, can require the accumulated experience of someone who has spent years doing the underlying research. This is not simply a checklist of facts an AI system can be trained on, it is accumulated knowledge that allows experts to recognize when something that looks convincing but is fundamentally wrong. “These are judgements that in some fields might only exist in a handful of human minds,” Windisch said.

This creates a problem that is easy to miss in discussions of whether AI can perform certain peer review tasks, because what are we asking AI to reproduce when we ask it to “assist” human judgement? And what might be lost if we automate a task without first fully understanding the expertise behind it?

Another conversation surfaced a different dimension of the problem, which is that peer review criteria are not fixed, but shift as fields evolve and their understanding of what counts as rigorous or valuable research changes.

“What is state of the art?” asked Anja Baethge, professor at Medical School Hamburg and editor for the journal Work and Stress. “As reviewers, we have so many quality criteria to assess a manuscript. But being an expert reviewer and defining ‘state of the art’ is not simply a matter of checking whether a paper meets each of these criteria. It’s deciding which criteria matter most and knowing how to weigh them.”

Baethge also points out that AI systems trained to reproduce today’s review standards risk becoming very good at enforcing the current paradigm while becoming correspondingly worse at recognizing work that legitimately breaks from it. “There would be no movement in our fields,” she said. In other words, the same criteria that make peer review consistent may be the same criteria that make it conservative.

There is a related risk in delegating the more “routine” parts of reviewing to AI. “AI does a good job of summarizing papers,” said Baethge. “But is it not the reviewer’s job to dig into the details?”

This is a different, albeit related, issue, where the concern is not simply whether an AI summary is accurate, because any summary is a negotiated transformation of meaning. It is a selection. And depending on the audience or the prompt, there will be differences in emphasis or implied importance. Two summaries of the same paper can be factually accurate while directing the reader’s attention to very different things. Even seemingly neutral outputs are interpretively structured, and each summary will differ in what it makes salient.

This matters because a reviewer does not encounter a manuscript in a vacuum. If a reviewer relies on an AI-generated summary, AI is not making the final judgement, per se, but it is helping determine what the reviewer notices before making it. The human remains in the loop, but the conditions in which that judgement takes place may already have been shaped by the system.

In a recent study published earlier this year in the Postgraduate Medical Journal, the researchers analyzed 398 peer review reports linked to 119 research articles published in BMJ Open in 2024. They found that human reviewers provided more detailed and thematically diverse feedback, addressing deeper issues such as interpretation and originality. AI reviews covered more sections of manuscripts but tended to focus on routine, structural, and formatting issues, where they performed slightly better. Their findings suggest that AI could be useful for screening basic errors and structural issues and highlights the differences in what human and AI reviewers tend to pay attention to.

We therefore come back to the original question, which is not simply whether AI can perform individual tasks better or faster than a human reviewer, but whether we understand the judgement those tasks sit within, and what happens when we begin changing the conditions under which that judgement is exercised.

The challenge may become even more pronounced as AI makes producing and submitting manuscripts easier. As the volume of research increases, the scarce resource may not be the ability to produce or process information, but the human capacity to decide what deserves attention.

A recent editorial published in Nature Nanotechnology makes a similar assumption in arguing that “reviewers and editors will be able to exercise their judgment in light of a vast amount of information in the literature that AI tools could retrieve effectively.” But what happens to the “human in the loop” when AI is also determining which sources a reviewer encounters, how those sources are summarized, and which aspects of a manuscript are brought to their attention?

This may be one of the less visible ways AI could change the “human” part of peer review because AI does not have to replace expert judgement to influence it. It can shape the information, comparisons, and possibilities from which that judgement is made, and this is what we risk if we automate editorial workflows before we have first understood what the human parts of those workflows actually involve.

None of this is an argument against innovation or against using AI for tasks at which it excels. Windisch, for example, says he could imagine AI being used to flag implausible data, identify suspicious patterns, or process quantities of information that would be difficult for a human to handle. But he drew a line around judgement. “At the moment, it must be done with humans,” he said. Experimental design, he argued, and the ability to judge whether a study genuinely addresses an important research question still depend on accumulated experience. “This is human, because you create something new.”

The goal, then, should not simply be to keep a human somewhere in the loop. It should be to understand what we mean when we refer to the importance of “human judgement” in peer review. Which parts of it can meaningfully be supported by AI? Which depend on forms of expertise that cannot easily be reduced to rules or retrieved information?

As the Nature Nanotechnology editorial cautions, “if used inattentively … we risk delegating critical thinking to an algorithm, giving us a false sense of achievement.” The challenge for publishers is therefore first understanding what, exactly, we are asking the human reviewer to contribute.