Editor’s Note: Today’s post is by Zeger Karssen. Zeger is the founder of World Brain Scholar, and leads a research and innovation team focused on developing AI solutions for the publishing industry.

Exponential growth in paper submissions, coupled with an increasingly overburdened and reluctant pool of peer reviewers, has created a bottleneck in the publication process. Publishing delays frustrate authors who are eager to see their work in print, and the situation has reached a critical juncture. Some researchers have even begun suing publishers over unpaid peer review work. Clearly, the time has come to rethink how peer review operates.

The Commodification of Knowledge

To understand how we got here, it’s crucial to look at the bigger picture. Knowledge is no longer pursued just for truth; it’s now a commodity. Science and technology drive economic growth, reflecting Enlightenment ideals which grounded modern science. This shift toward “cognitive capitalism” has industrialized and globalized research, pressuring scientists to publish. What started in 17th-century scholarly societies is now a multi-billion dollar industry, with millions of researchers under pressure to publish.

As competition grows, academic publishing has adopted business strategies focused on efficiency and profit, making it resemble other global industries. Yet, peer review has remained an artisanal part of this system.

A System Under Strain

Peer review, essential for quality in academic publishing, depends on experts to assess a paper’s originality, methods, and contribution. Scientists contribute over 130 million hours annually—equal to 15,000 years of labor—yet the process remains informal and inconsistent, with few clear standards. Most reviewers receive no formal training or compensation, and the majority of reviews come from Western countries.

As submissions surge and recognition and support for peer reviewers is scarce, reviewers are increasingly reluctant to participate. Authors are less and less accepting of long publication delays where peer review remains the main bottleneck. And although some publishers have drastically reduced review times, this often came at the cost of quality.

So, is it any wonder, then, that peer review is in crisis?

Principles for Reform

We’re at a breaking point, and peer review needs quick reform. One solution is better training and compensation for reviewers, something we will not elaborate in this blog.

Let’s focus on a recent solution: AI technology (full disclosure, my company, World Brain Scholar offers AI solutions including a peer review assistant). Advances in AI now allow tools to analyze content, assist reviewers in drafting reports, and aid in decision-making. Leveraging Natural Language Processing (NLP), these tools enable intelligent, natural interactions and help manage large volumes of information. They can act as human-like assistants, providing valuable feedback for both reviewers and editors. Key NLP technologies needed for supporting peer review include:

• Large Language Models (LLMs): These systems, trained on extensive text data, understand and generate human-like language by recognizing statistical patterns. In peer review, LLMs can assist with text analysis, summarization, and highlighting important aspects. Paper-aware tools, like chatbots and interactive forms, use LLMs to collaboratively draft review reports in real-time. While general LLMs like ChatGPT handle some peer review tasks effectively, a dedicated LLM trained on large peer review-specific datasets is needed for optimal support.
• Retrieval-Augmented Generation (RAG): This technology merges information retrieval with language generation. RAG retrieves relevant documents or data from various sources and uses this information to create more accurate, context-aware responses. In peer review, RAG sources content in the original paper and review reports, making review comments traceable and justifiable.
• Semantic Profiling (SP): This technology analyzes the meaning and context of text to identify key themes, topics, and relationships. Going beyond simple keyword matching, SP offers a deeper understanding of content. Tools using SP can assess similarities between papers and other texts, aiding in literature retrieval, reference relevance checks, and more.

Having AI available is one thing, but designing an effective peer review tool with this technology requires adhering to several key principles, including:

1. Supportive: The AI tool should assist reviewers by offering suggestions for improvement while allowing them to retain full control over the final output.
2. Engaging: Reviewers should be able to interact with the tool to actively construct their reports, maintaining their engagement throughout the process.
3. Responsible: The AI tool must uphold the integrity of peer review by providing detailed feedback and helping reviewers identify potential issues.
4. Trustworthy: The tool’s suggestions should be based on high-quality, relevant datasets to minimize inaccuracies and ensure that recommendations are data-driven and reliable. The ability to trace the sources behind the tool’s suggestions is crucial.
5. Privacy and Compliance: Adherence to regulations like GDPR and the EU AI-Act is essential to ensure that privacy and copyright concerns are addressed.
6. Transparent and Accessible: The AI tool should generate clear, actionable feedback and be intuitive for users, with language support that broadens the global pool of reviewers.

AI tools based on these technologies help reviewers write more comprehensive and clearer reports, enhancing quality and transparency. They also expand the reviewer pool by reducing language barriers and providing real-time guidance to less experienced reviewers. For editors, these tools quickly summarize key feedback from multiple reviewers, facilitating informed decisions and actionable feedback for authors.

Outlook

AI tools are set to revolutionize academic publishing. While we’ve focused on their impact on peer review, the near future will see AI shaping pre-review technical checks and even the drafting of scientific papers, ensuring that submissions are ‘scientifically sound’ and require only a final journal-specific review before publication. Publishers that invest in these technologies will drastically reduce writing and review times while improving quality, objectivity, transparency, and trust.

Looking further ahead, AI already has the potential to fully automate scientific discovery, from developing and testing hypotheses to producing peer-reviewed papers ready for publication. There’s no doubt that AI will continue to revolutionize science and publishing in the coming years. Publishers must develop strategies on how to integrate AI into their operations to remain competitive and protect their very ‘raison d’être.’