In November it will have been three years since ChatGPT was launched, and what a couple of years it has been. ChatGPT has changed the way we think about generative AI, both as a sector and as a society, with its accessible, conversational interface that enables anybody with a web browser to interact with it. In other words, it took the progress that computer scientists have been making for decades and made it directly visible to the public.

It’s no surprise then, that the Gartner hype cycle put generative AI right at the pinnacle of the peak of inflated expectations in 2023. As with any innovation, there was both excitement and skepticism in those early days. There were, and still are, strong concerns around attribution, the risk to copyright, hallucinations, low-quality content overload (aka slop); some even saw it as an existential threat to creativity itself. I acknowledge these risks, but I’m not overly concerned. Fear of risk shouldn’t prevent us from experimenting and innovating, instead they should be carefully monitored and mitigated. For example, privacy and security risks can be mitigated through approaches like guardrails, while risks posed by functional issues like hallucinations can be tackled by choosing the right technology for the right problem, for example retrieval augmented generation (RAG), which enables LLMs to incorporate new information in their answers without retraining as well as directing users to the sources on which claims are based.

From an innovation point of view, a more interesting question is whether we’ve really seen use cases that measure up to the huge amount of effort and expense that have gone into building the LLMs that power many generative AI applications. In a post I wrote about a year ago, I wondered out loud when the crash in the hype would come and, assuming things follow the hype cycle model, what would lie on the other side of the so-called trough of disillusionment.

While I’m still not sure when all this AI excitement will turn in on itself, or how deep the disillusionment will go, I find myself wondering whether, a year later, we know any more about what this technology will eventually do for us?

How have things evolved over the last year or so?

Early predictions of the role of generative AI in academia and publishing focused on productivity, research, and writing support. Making writing easier and quicker is an obvious use case for something called a language model, but concerns have been raised about the risks to academic integrity and cheating, leading to what seems like something of a split in this part of the market. On one side, there are tools — which I won’t name here — marketed as being able to do your homework for you and not get caught by AI detectors. However, others have taken a more ethical approach. Grammarly and Writeful, who I mentioned in my post last year, have continued to develop responsible AI tools that help users improve their own writing, rather than doing it for them. We’re now starting to see tools that act more like AI-powered writing coaches that don’t necessarily generate your text directly, like Thesify, with its writing critiques and suggestions of what literature to read, and Profectus, which specializes in grant proposals and also offers services like training workshops on how to get funded.

Discovery and summarization tools were also among the first real product types to emerge, and now pretty much all of the web-scale discovery services have their own AI-powered natural language search and discovery interfaces. Meanwhile, startups like Elicit, ResearchRabbit, Scholarcy, Scite, Perplexity, and Proemial, among others, continue to make progress refining their target markets and use cases. One interesting area is AI summarization for specific audiences. Most of the examples I’ve seen so far are intended to make content more accessible for the general public, students, or researchers in an adjacent field. Solutions that are compellingly tailored to business applications seem to be still maturing. One example is Boardly, which is an early tool designed to help in the creation of board materials, and there are also products like Google’s NotebookLM, and QuillBot’s Executive Summary Generator. In the future, I think we’ll likely see more offerings like Iris and Questel, which integrate with external and internal documents and data, combining AI agents with technologies like RAG to help leaders make good strategic decisions without getting bogged down in information overload.

Automated Scholarly Paper Review (ASPR) is a whole new field that’s exploding the use of AI to protect research integrity. It includes tools for editorial triage, event management, formatting, scope checks, and reviewer matching. Clear Skies and Signals have received the most attention over the last year or so, with their network analysis approach to detecting patterns of research fraud. There are many fraud detection and integrity check workflow tools on the market, many of which focus not on network analysis, but on detection at the point of review. The sheer number of them may be evidence that the barrier to entry for building a workflow tool may not be very high. Examples include Morressier’s Effortless peer review, Editorial Pilot from Integra, Peer submit, and startups like Penenlope, reviewer zero, reviewer one, reviewer3 (is anybody noticing a pattern, here?), and newcomer nicia.

With so many options, it’s easy to feel overwhelmed, and with many of these tools being stand-alone products, they need to be integrated into existing workflows without unduly increasing the time needed and the cost of screening manuscripts. Several approaches have emerged to meet this challenge. The STM Integrity hub continues to bring tools together on a shared platform. Offerings like ReView 3.0 from River Valley Technologies provide integrated publishing workflow solutions, while Molecular Connections’ Research Integrity Toolkit focuses on integrating tools with existing platforms. Meanwhile some larger commercial publishers like Springer Nature and Wiley are partnering with technology companies to integrate ASPR directly into their own platforms. 

Where are things heading?

It’s clear that there is a lot of activity around AI innovation in publishing. The early trends in discovery, summarization, and workflows, particularly with editorial and research integrity, are still proliferating but are now showing signs of maturing and becoming more integrated. 

It’s still hard to see what will happen in the longer term, but one popular idea, as articulated recently by Eric Schmitt, ex-CEO of Google, is that user interfaces will go away, to be replaced by AI agents. What that looks like for scholarly publishing and for academia remains to be seen. Perhaps the trend towards expanding publisher services upstream will continue beyond author tools and we’ll eventually see AI agent-driven research and dissemination workflows all the way from the lab bench to the readers’ screen. Maybe the form of research publication itself will finally change and we’ll see agentic publications where a body of work, submitted as something like a research object, can be built on, interrogated, or synthesized along with other research through a natural language interface. Perhaps that will inevitably lead to the long-promised disruption to how research and researchers are evaluated and incentivized. Who knows? All I’m willing to say confidently at the moment is that the workflow and efficiency enhancements that AI currently provide are likely to continue to gain pace in the short to medium term and we still don’t know where this journey will eventually end.