Editor’s Note: Today’s post is by Tony Alves, scholarly publishing professional since 1990, currently serving as SVP of Product Management at HighWire Press.
My first Peer Review Congress was in Barcelona in 2001, an auspicious and memorable meeting due to the 9/11 attacks. I’ve been to most of them since then and have seen a lot of changes in the topics presented and what resonates with the audience. Comparing the two most recent meetings in 2022 and 2025 has been particularly interesting. When I attended the 2022 Peer Review Congress, the conversation around artificial intelligence (AI) in peer review felt tentative with a touch of anxiety, and with folks unsure of AI’s potential. It was seen mostly as a promising tool to streamline discrete editorial functions. AI was treated as a supplementary aid, not a transformative force.
This year, at the 2025 Congress, the tone has shifted. Across plenaries, abstracts, and side conversations, I sensed a more reconciled, yet apprehensive, attitude toward AI’s accelerating influence in manuscript preparation, reviewer behavior, journal policy, and research integrity. If 2022 was about the promise of AI-enhanced tools, 2025 was about grappling with AI’s pervasive, unpredictable, and often opaque integration into the peer review ecosystem.
The 2022 Congress offered a few explorations into the use of AI in scholarly publishing, most notably in Daniel Evanko’s study of Proofig, a commercial tool used to detect image duplication in submitted manuscripts. His presentation underscored AI’s practical utility: screening for image reuse was faster and more effective with AI than manual checks. He showed that most duplications were found to be unintentional, pointing to poor data hygiene rather than fraud. This helped set the tone for how AI was largely viewed at the time, as an enhancement to editorial diligence.
Isabelle Boutron from Université de Paris also floated the idea that peer reviewers could benefit from “useful AI tools” to help detect spin and bias in manuscripts. But this suggestion was part of a broader examination of reviewer fallibility and unrealistic expectations of peer review. The emphasis was still on human judgment, with AI considered a supportive but limited resource.
By 2025, the conversation has evolved. Several studies presented at this year’s PRC explored not just whether AI was useful, but how deeply it had already embedded itself in scholarly writing and review, often without sufficient transparency. A recurring theme was disclosure, or the lack thereof.
Disclosing the Use of AI
A standout presentation came from Isamme AlFayyad and colleagues, who analyzed 25,114 submissions to BMJ Group journals after a mandatory AI disclosure policy was introduced. Surprisingly, only 5.7% of manuscripts disclosed AI use, despite external surveys estimating usage rates as high as 76%. Most disclosures cited generative chatbots like ChatGPT, used primarily to improve the quality of writing. Yet even with policies in place, many researchers failed to disclose. Underreporting was most notable in Oceania (1%), while Europe saw the highest disclosure rates (31%).
The tension between policy and practice was even more apparent in Evanko’s new 2025 study, Quantifying and Assessing the Use of Generative AI by Authors and Reviewers in the Cancer Research Field, from the American Association for Cancer Research (AACR), which combined self-reported and AI-detected data to assess AI’s presence in over 46,000 manuscripts and nearly 30,000 reviewer reports. The results confirmed suspicions: by 2024, 23% of abstracts contained AI-generated text, yet fewer than 25% of authors disclosed using AI. Also, papers flagged as AI-generated were significantly less likely to be sent out for peer review, showing institutional bias against stated AI usage.
Authors from non-English-speaking countries were twice as likely to use generative AI, showing AI functions as a linguistic equalizer. AI use among reviewers was smaller but growing. Evanko’s data showed a drop in AI-generated reviewer comments after the AACR implemented a prohibition policy in Q4 2023, but usage resumed an upward trajectory soon after. There was a lot of skepticism from the audience that the AI detection was accurate, but I will take the study at its word.
Zak Kohane of NEJM had one of the most exciting papers. In his study, A Singular Disruption of Scientific Publishing — AI Proliferation and Blurred Responsibilities of Authors, Reviewers, and Editors, he argued that AI is not merely creeping into the publishing process but will fundamentally reshape it within the decade. Kohane predicted that AI reviewers would soon surpass the majority of human reviewers in quality. He described NEJM’s experimental “AI Fast Track,” where submissions were evaluated by both human and AI reviewers (including GPT-5 and Gemini Pro). In some cases, the AI reviewers flagged issues that humans overlooked, suggesting a future of professional “AI-augmented reviewers.” Yet, Kohane cautioned against techno-utopianism: systems remain vulnerable to manipulation. He insisted that human oversight must remain in the loop, even as AI improves. I see the inevitability of AI’s integration into the peer review process, and we have yet to see if AI strengthens or weakens the trust infrastructure of scientific publishing.
The disclosure issue was highlighted by the JAMA Network team (Perlis, Flanagin, et al.), which examined 82,829 submissions across 13 journals. Only 2.7% disclosed AI use, most commonly for language editing. Reviewer disclosures were even rarer. Reviewers who acknowledged their use of AI were rated slightly higher in quality, showing that perhaps the stigma of AI use drives the reluctance to disclose.
A China-based survey of 159 peer reviewers confirmed both high rates of AI adoption and chronic underreporting. Nearly 60% had used AI, mostly for translation and language polishing, but only 29% had declared it. Early-career researchers, especially those under 30, were most likely to use AI. Again, the theme was clear: AI is already deeply embedded in the writing and review process, but acknowledgment lags far behind.
From Policy to Pragmatism
While much of the discussion was data-driven, there was also an ethical and cultural debate among the attendees. What constitutes appropriate use of AI? When does language support veer into substantive content generation? Should AI use affect acceptance decisions? And how should journals balance transparency, accountability, and inclusiveness when setting policy around AI?
Several presenters touched on the practical benefits AI offers early-career researchers and scholars writing in English as a second language. For these groups, AI tools are not shortcuts; they are ways to gain equal footing. Evanko’s data confirms that non-native English-speaking authors are more than twice as likely to use AI, especially for abstracts. Yet, these same papers are more likely to be rejected before peer review. Is this bias, or is it quality concerns?
Then there is the question of accountability. In the Drummond Rennie Lecture, Ana Marušić revisited the debate over authorship vs. contributorship and asked whether AI tools should ever be acknowledged as co-authors. Her answer was a firm no, in line with current editorial norms. But she acknowledged the blurring lines of agency and contribution in an era when generative AI is increasingly indistinguishable from human prose.
The evolution of AI in peer review between the 2022 and 2025 Peer Review Congresses is not just about technology adoption; I also see it as a discussion about cultural transformation. What started as a conversation about useful tools has now become an ethical, editorial, and operational concern for the entire scholarly communications ecosystem.
In 2022, AI was seen as an efficiency tool for image analysis and language polishing. By 2025, AI has become a participant in the process, shaping writing, influencing review, and challenging the concept of authorship and accountability. The gap between use and disclosure shows a community still struggling to normalize transparency.
The 2025 Peer Review Congress made it clear: The future of scholarly publishing will depend not only on technical solutions but on collective choices about accountability, equity, and trust. I no longer see a debate as to whether AI will shape peer review. Kohane’s presentation revealed a future where AI reviewers outperform most humans, where journals like NEJM are already piloting AI peer review, and where decisions made in the next two years will shape the trust infrastructure of scholarly publishing. His insistence that “culture trumps technology” is a reminder that even as AI spreads, the norms, values, and governance frameworks we build around it will determine whether it strengthens or undermines science.
Discussion
2 Thoughts on "Guest Post – How the AI Debate Has Changed in Just a Few Short Years"
This is a brilliant summarisation of artificial intelligence use throughout the scholarly comms milieux, Tony. Thank you.
I was annotating your article and when I got to your conclusion, everything I was writing in my notes was beautifully summed up in your final paragraph! The use of generative AI is, definitely, a matter of cultural – and potentially tribal – explicit or otherwise, “rules” that will arise to ensure Trust in the system; particularly by those within that culture/tribe. These “rules” may well be different across publishers, research cohorts, journals, libraries and educators, but in every instance, transparency will be a requirement to breed trust in this new process.
Authors, publishers, researchers have been using new technologies to communicate for centuries (the pen, printing press, computers, interwebs et al..), but genAI is a very different beast because of its power to mimic human language and cognition. It means we can’t be sure anymore, about what is, and who, are behind the curtain: ergo, that intuitive human understanding we culturally determine from a text, about motivations and purposes.
Accountability, equity (& I will argue that non-English speaking researchers utilising AI – ethically – to get their work onto the same stage as the rest of us, is a pretty good use of AI)… and Trust are now absolutely, the currency that predicates the use of AI in research and communicating research. Transparency in this case, will mean a confirmation for meeting standards of Accountability, Equity and Trust. It may require explicit rules and definitely a reconfiguration around provisions of those proofs.
AI in scholarly communication is going to be used and probably extensively. It will only really be useful insofar as people trust where it’s come from, why it’s been utilised, who has used it and to what ends and whether the rules within the culture deemed that use need appropriate. That’s shifting sands, right there, depending on the culture or the tribe!
Therefore, transparency is the actual new paradigm here. For the first time in human history, we need to explain – even argue – at every point, why we’ve used a particular technology in the communication of our work. We’ve never had to do that in any serious way before. It’s a huge leap and we’re right now, crawling before we can walk this new path.
Note: No generative AI was used to write this comment 😉
The disclosure gap you highlight—76% usage vs 5.7% reporting—reveals something deeper than just policy compliance issues. It suggests we’re in a liminal moment where AI has become so normalized in daily research workflows that many don’t even recognize it as ‘AI use’ anymore (or, is that people know that they can get away not disclosing AI use?).
The real question isn’t whether AI will transform peer review, but whether we can build governance frameworks fast enough to preserve trust while embracing inevitable efficiency gains. Kohane’s ‘culture trumps technology’ insight is crucial—we need social norms to evolve alongside the tech.
