Editors’ note: Today’s post is by Steve Smith and Josh Nicholson. Steve is VP of Publisher Licensing & Commercial Partnerships at Research Solutions. Josh is Chief Strategy Officer at Research Solutions and co-founder of Scite.

The Disappearing Download

A researcher asks an AI assistant, such as ChatGPT or Claude, to assess the evidence for a scientific claim. A few minutes later, a two-page answer appears, with three citations. The researcher clicks one link, looks at a figure, and goes back to work.

Now, suppose we could reconstruct everything that happened behind that answer. In this hypothetical case, after an initial search, the assistant requested the full text of thirty-two papers. Two requests were refused, although their abstracts remained available. It retrieved passages from the remaining thirty. Passages from six of those papers were supplied to the AI model as evidence it could draw upon, and three papers were cited. 

How many times was scholarly literature used? Thirty-two requests, thirty retrievals, six sources supplied as evidence, three citations, one click by the reader. Depending on the access route, a conventional publisher usage report might capture some of those events, or none. 

Each is a different measure of use: demand, access, potential contribution, attribution, or human engagement. Even a complete record of these events would not tell us exactly which sources shaped the answer, or whether the researcher found it useful. 

Decorative image: styleAbstract colorful digital landscape with flowing particles. Cyber or technology background. Red, pink, orange colors.

Scholarly publishing has spent decades building standardized measures of usage, most notably through COUNTER. Article requests and investigations were always imperfect proxies for reading, but they provided a scalable, comparable record of researchers’ interactions with scholarly content, without requiring the monitoring of individual readers. AI assistants generate activity those measures were never designed to capture. 

The obvious worry is that usage is migrating into channels publishers and libraries can’t see. Some of it may well be. But a decline in downloads does not establish where that activity went. The same pattern could reflect changes in research practices, the mix of researchers, alternative access routes, or genuinely diminished reading. Distinguishing among these possibilities requires more than two trend lines. 

The industry is beginning to build the necessary evidence through an increasingly coordinated effort. COUNTER’s new AI reporting guidance draws on the SPUR Coalition‘s telemetry framework, while NISO is piloting a minimum metadata model for provenance and attribution. The recently launched TRACE initiative is helping coordinate these and related efforts to make AI-mediated scholarly research more transparent and measurable. 

As these new measures of AI-mediated activity find their way into publishers’ usage reports, a more fundamental question arises: should everything an AI assistant does with scholarly content count as usage in the same sense? We don’t think so. We propose four overlapping ledgers for interpreting them: usage, demand, contribution and engagement. Each illuminates a different kind of value, often for a different party, and their totals should not simply be added together. 

Five Kinds of AI-Mediated Activity

With an AI assistant in the middle, what used to look like a single content request can now involve several distinct activities. The SPUR Coalition’s Content Telemetry specification identifies five types of event: retrieved, grounded, cited, presented and engaged. COUNTER’s October guidance explicitly maps these events onto its reporting framework. SPUR describes the activity occurring within an AI workflow, while COUNTER translates selected events into standardized, comparable usage measures. 

  • Retrieval means content fetched from a server, marketplace or index. That might be an abstract, excerpt, or full-text passage. The assistant has expressed interest in the material, but may never use it to compose an answer. 
  • Grounding occurs when content enters the AI model’s generation context. This is more specific than a fetch: the material could inform the answer. It does not establish how much influence the source actually exerted. 
  • Citation records an explicit association between an identified source and an AI output. It does not guarantee that the source supports the adjacent claim. 
  • Presentation means content or a source reference was made perceptible to a recipient, such as a link or citation card in a response. 
  • Engagement records an observable action, such as clicking a link, copying text or sharing a result.

These events do not always follow a tidy sequence. A source may be presented without being cited, cited without a matching grounding event, or grounded from a cache without a fresh retrieval. 

The temptation is to add these events into one figure for “AI usage.” We think they belong in separate ledgers, because the same event can mean different things to different parties. 

What Can Be Counted, and Who Can Verify It?

As Tasha Mellins-Cohen and Michael Sisolak explained recently in Research Information, COUNTER’s phased approach deliberately distinguishes meaningful use from the broader range of AI retrieval activity, while drawing on SPUR’s framework for reporting both. 

COUNTER’s Best Practice on Generative and Agentic AI Usage Metrics, updated on October 23, 2026, counts excerpts an AI system retrieves even if they are never selected for an answer, so delivery and use are reported separately. 

In COUNTER’s mapping of SPUR events, a retrieval contributes to retrieval metrics; a cited or presented item may contribute to an AI Investigation, and to an AI Request when full text is accessible. COUNTER does not count grounding on its own. An assistant might retrieve 10 excerpts from one article, ground two, and cite it once. A thousand excerpt retrievals do not represent a thousand articles, much less a thousand human readers. 

Who can see those events? A properly instrumented publisher gateway can record what it served, when access was withheld, and, where authentication is available, which institution made the request. Beyond its own servers, a gateway sees only what connected services choose to send back. 

SPUR offers a way for participating systems to exchange events and corroborate some retrievals against content-provider records. Standardizing how an event is reported does not prove that every occurrence has been captured. 

COUNTER’s syndicated-usage guidance permits institutional reporting when authentication or other identifiers match exactly. For consumer accounts, anonymous agent traffic or activity outside an institutional entitlement, attribution may be unavailable. COUNTER explicitly rules out best-guess institutional matching. 

NISO is piloting a minimum metadata model for provenance and attribution, while TRACE, the Trusted Retrieval and Attribution for Content Ecosystems initiative, coordinates work on content access, provenance and reporting. Its access workstream, an Ithaka S+R study sponsored by publishers through STM, is examining how AI agents discover scholarly content, authenticate through user entitlements and leave a credible record of use. 

These initiatives are providing much-needed common definitions and reporting infrastructure. COUNTER offers an established foundation for standardized, independently verifiable usage reporting, although its new AI metrics remain optional and are not yet subject to formal audits. Platform-specific data can complement that foundation, provided their definitions, coverage and limitations are clear. Standardizing what happened still leaves the interpretation: what the activity means, and who benefits from it.  

Whose Value Is It?

Suppose the researcher’s library sees downloads from a journal collection fall by 15% over a year. At renewal, the publisher reports thousands of additional AI retrievals and presents them as evidence that the collection is still earning its price. Should the library accept those numbers on those terms? 

Folding everything into one usage figure can obscure who benefited, what was delivered and who might reasonably be expected to pay. The four ledgers keep those questions apart. They are categories for interpreting value, and they build on COUNTER’s metric definitions.  

Usage. The first ledger records content successfully delivered for a researcher’s task, including AI-mediated access under an institution’s existing subscription. Some of the thirty retrievals in our example might fit this description. They belong in renewal evidence where they can be attributed to the institution, but they should be identified as AI-mediated events. COUNTER’s Agent access method, an optional extension that a library can request, lets it inspect them apart from human downloads, instead of receiving a combined total that implies the two are equivalent. 

Demand

The second ledger records what AI systems seek, whether or not it appears in an answer. Our assistant requested thirty-two papers, retrieved thirty and was refused full text for two. Those requests show what the assistant went looking for and where access failed. 

For publishers, demand is useful intelligence. It can guide discoverability work and identify gaps in access. Refusals are especially intriguing, although a refusal through one route does not prove that the researcher lacks a subscription elsewhere. 

One question that remains open is how to report unsuccessful AI access attempts. COUNTER has established denial measures for conventional usage, but its October guidance does not yet define an equivalent specifically for Agent access. Usage logs from Scite (with which both authors are affiliated), excluding internal and test accounts, offer an indication of what is missing: from October 7 to November 6, 2026, assistants attempted 7.2 million reads of publisher content through Scite, of which 3.7 million were refused full text because AI-access permission was not available through that service, a refusal rate of 51%, far higher than our hypothetical example suggests.

Those refusals include some open access content: Scite does not supply articles under CC BY-NC or CC BY-ND licenses to AI assistants. The figures are gateway-specific attempted reads, not unique researchers, audited COUNTER denials, or evidence of lost sales. They nevertheless record a form of demand that should be visible separately from successful usage, with a clear indication of why access was withheld. A refusal for want of AI-use permission is also a transaction that could not happen: in most cases the agent has no machine-readable way to ask for rights at the point of need. 

In our hypothetical example, twenty-four retrieved papers never entered the model’s generation context. Presented as twenty-four article readings, those events tell a librarian considerably more about the assistant’s search strategy than about value delivered by an existing subscription. Agentic search, which fans one question out into many queries, will widen that gap. 

Contribution

The third ledger is where things become more difficult. Six sources were supplied to the assistant as evidence it could draw upon, and three were cited. This activity offers evidence about the scholarly material available to support an AI-generated answer. 

Real-world data offer a glimpse of how AI assistants make these choices. In October 2026, Scite-connected assistants that reported their citation decisions recorded 1.36 million decisions about whether to cite or set aside a source. Of these, 57% were reported as cited. These are the assistants’ own accounts of their decisions, and only some sessions provide them. The figures therefore cannot be generalized to all Scite retrievals, much less to other AI services. Nevertheless, they demonstrate that decisions about which sources to cite can be recorded separately from retrieval activity. This is consistent with the distinction COUNTER’s new guidance makes between initial retrievals and the content subsequently selected for an answer. 

The value of scholarly content in these workflows can accrue to more than one party. The researcher may receive a better answer, potentially enabled by an institutional subscription. The AI provider may deliver a more useful service because it can draw on reliable scholarly evidence. The benefits overlap, but the relevant rights and commercial relationships may differ. 

That leaves two questions rarely asked together. If grounding and citation figures help justify subscription prices, libraries are entitled to ask what value their researchers received. Publishers are equally entitled to ask whether a commercial AI service is deriving additional benefit from licensed content, even when no model training occurs. In our view, value captured by an AI provider deserves consideration in the publisher–AI-provider relationship. Whether it calls for another payment depends on the rights already granted and the service actually provided. 

Contribution is also hard to trace. An assistant can reuse material from an agent-side cache or stored research context without a fresh request to the publisher. An answer may also rely on a review that summarizes an earlier study, giving the study an indirect influence without a direct retrieval or citation. Counting requests will not settle this. That will take evaluations and shared benchmarks that show whether an answer still reflects its source, including any corrections or retractions, once the synthesis is done. 

One approach is what Todd Toler and Angela Cochran called, in The Scholarly Kitchen (March 2026), “subscribe-to-context”: licensed, metered, attribution-preserving access to scholarly content, supplied to AI systems as context at the moment of inference instead of being absorbed into a model’s weights. Such arrangements could recognize the value of scholarship to an AI service separately from the value an institution derives from its subscriptions. Each would also need an account of what was delivered, to whom and for whose benefit, which is what the ledgers are for. 

Engagement

The fourth ledger records what the researcher actually does. In our example, that was one click. For a journal or society trying to sustain a relationship with its community, this may be a valuable moment: the researcher encounters the source itself, and perhaps the people and wider scholarship behind it. 

The absence of a click isn’t failure, though. The researcher may have got exactly what they needed. One measure of a good AI assistant may be how rarely its users need to leave the conversation, which is an awkward fact for any publisher whose strategy depends on them arriving. 

Separating the ledgers exposes a tension. Publishers have incentives to demonstrate the largest defensible volume of AI activity. AI providers hold much of the information about downstream grounding and citation, and may have commercial reasons to limit disclosure. Libraries have to make collection decisions using those reports. A headline total with opaque counting rules invites the skepticism librarians already bring to vendor dashboards. That skepticism applies to platform-specific metrics, including Scite’s: however carefully built, they lack the independent audit and comparability that a shared standard such as COUNTER provides. 

There is a fair objection: libraries license content to support research. If an assistant gives a better answer because it can reach a university’s subscribed literature, the library has received value, whether or not the AI provider benefited too. 

We agree, and that is why each kind of value should stay visible to whoever is making the decision. 

Keeping four ledgers is a proposal about accounting. Who invoices whom is a separate negotiation, and the accounts should be clear before it starts. 

What Would Credible Reporting Look Like?

If AI-mediated activity is to become evidence that publishers and libraries can act on, reporting has to keep the four ledgers distinguishable. 

Events need consistent counting rules. The report must specify its unit: excerpts, items, answer-level references or user actions. Otherwise differences among providers may reflect their technical architecture more than differences in research use. 

Events also need to identify the underlying scholarly content. A DOI connects a citation to the Version of Record, but when a service works from a preprint, a corrected article, a stored excerpt or a derived passage, a report should identify the source and version where those distinctions are material. NISO’s revised Journal Article Versions Recommended Practice and its separate provenance pilot offer useful groundwork. 

Each observation needs enough context to put it in the right ledger. Was full text available, or only an abstract? If full text was withheld, was the obstacle an absent subscription, a separate AI-use permission, or a limitation of the service itself? Which service generated the request, and did authentication establish an institution? 

Reports should also say what they cannot see. We would like every AI usage report to carry a coverage statement: which services are included, which events each can observe, how the counts are derived, and where the gaps remain. Independent verification, where technically possible, would add confidence. 

These are requirements that publishers and institutions can begin negotiating now, alongside the standards work. Where an AI service seeks permissioned access to scholarly content, event definitions, reporting coverage and verification arrangements can be agreed and tested in a pilot. The specific coverage statement proposed here would complement, rather than replace, COUNTER and SPUR. If publishers and libraries do not establish their expectations, vendors will define the defaults. 

Beyond the Download

Return to our hypothetical researcher. How many times was the literature used? 

The four ledgers give four answers. Usage is whatever share of the thirty retrievals was supplied under the library’s subscriptions and can be attributed to it. Demand is thirty-two, including the two refusals. Contribution is six grounded sources and three citations, with their precise influence unknown. Engagement is one click. None of these counts substitutes for the others. 

COUNTER, SPUR, NISO and TRACE have made real progress toward measuring activity that conventional publisher reports rarely captured. The work ahead is to improve coverage and keep successful use distinct from unserved demand. 

The risk is that new numbers get added together and presented as the successor to the download. Kept distinct, they can help libraries assess what researchers receive, show publishers where demand lies and inform fairer commercial relationships with AI services. Added together, they will mostly tell each party what it hoped to hear. 

Authors notes:

  • Research Solutions participates in NISO’s provenance and attribution project.
  • We worked with Claude to summarize and organize notes from our discussions, and as an editorial aid in reviewing drafts and copyediting. The ideas, arguments, source selection, and final decisions are entirely the authors’ own.
  • We thank Chris Bendall, Tasha Mellins-Cohen, and Todd Toler for their comments on an earlier draft of this piece. 
Steve Smith

Steve Smith

Steve Smith, DPhil, is Vice President, Publisher Licensing and Commercial Partnerships at Research Solutions.

Josh Nicholson

Josh Nicholson

Josh Nicholson is the Chief Strategy Officer of Research Solutions and co-founder of Scite. Previously, he was the founder and CEO of the Winnower and CEO of Authorea, two companies aimed at improving how scientists publish and collaborate. He holds a PhD in cell biology from Virginia Tech, where his research focused on the effects of aneuploidy on chromosome segregation in cancer.

Discussion

Leave a Comment