Editor’s Note: Digital delivery of scholarly publications has enabled far more robust tracking of usage, with the COUNTER Project providing and periodically updating the defining standard for usage measurement. As a result, usage has become a critical metric for establishing the value of a given journal or content bundle in many circumstances, including licensing negotiations between publishers and libraries. This has caused one Scholarly Kitchen author to wonder, Are Library Subscriptions Over-Utilized? At the same time, concerns about usage “leakage” from the publisher platform to other services where publishers have not received “credit” for that usage has led to efforts to re-enclose that usage through syndication.

Against this context, Curtis Kendrick, Dean of Libraries at Binghamton University and a key leader in SUNY’s collective licensing initiatives, has raised some probing questions about whether cost-per-use is the appropriate metric for measuring the comparative value of library subscriptions. Today’s piece offers a strong warning, to publishers and libraries alike, to avoid the simplistic use of metrics when the underlying thing being measured is far more complex.

The subscription monopoly has been broken. Content is a commodity; it is ubiquitous.  What we are paying for by subscribing to journals is as much convenience now as it is access, and the valuation paradigm has to change. The cost per use model is too simplistic a measure as it does not account for variability in the nature of patron usage, consequently overvaluing journal subscriptions. Cost per use fails to take into consideration variability in the nature of usage – not all usage is equal, not all usage has equal value. Usage by a sophomore who happens to download an article from a high-end database is qualitatively different from that of a research scientist racing to complete an application for an NSF grant. In the former case, the undergraduate who downloads an article from a high-end database may be served just as well as by an article from a less costly substitute information resource. In the latter, a research scientist’s close reading of the article from the higher priced resource may be the difference between winning and losing the grant. In this piece, I critique the limitations of the “cost per use” valuation exercise and provide some alternative ways to approach resource valuation.

With limited resources, one of the ways in which we can make our dollars go further is to segment our market (users) and do a better job of providing what they need rather than providing a lot more than what they need. In negotiations between libraries and vendors, the strategy of basing the assessment of value on cost per use is advantageous to the vendor. Because cost per use has become a de facto standard, there is a common measure of value by which vendors can compare usage and spending patterns at different libraries and use this information to their advantage as leverage in negotiations. Libraries typically do not engage in such comparisons of cost per use against other libraries, or do so on a very limited basis because of our uncertainty about non-disclosure agreements. As long as cost per use is the generally accepted metric for value, vendors will have an edge in assessing how important titles are (or, they will postulate, should be) to the library.  The more self-sufficient libraries can become at assigning value the better.

Libraries have an opportunity to seize greater control of the negotiation conversation by reducing the substantial informational asymmetry that privileges vendors. They must conceive of a different scale for evaluating journal subscriptions. Whereas an advantage of the cost per use model is that it is relatively easy to calculate and easy to understand, it doesn’t get us too far from our practice of mistaking “counting things” for assessment. While our industry norm has been to think in terms of the cost of the journals we license, what if that cost were to be viewed instead as an investment? Less, “how much does this journal cost?” and more “what are we getting for our investment in this journal?” What is our ROI, our return on investment? We need to shift perspective to focus on the investment side of the equation and understand not only what are we getting for our money, but also what is our money is getting for us? In other words, for each dollar invested how much usage do I get? While the model will get more complicated in a moment, think of it essentially as shifting from Cost Per Use (CPU) to Use Per Cost (UPC).  Either way, for collection analysis purposes we would end up drawing the same conclusions as the table below suggests.

Table 1: Comparison of Cost Per Use (CPU) and Use Per Cost (UPC) Analysis

In the example above Resource A has a cost per use of $2.00 and Resource C has a cost per use of $10.00. Resource C is five times more expensive. Similarly, for each dollar of our $50,000 invested in Resource C we get 0.1 uses, while for Resource A’s $10,000 investment we get five times as many uses per dollar. Making the flip from  cost per use to use per cost allows us to answer the question “for every dollar invested in resource X how many uses do we get?” This investment-based approach is better suited to library planning because it keeps cost constant, but as with cost per use, can allow us to vary the usage calculation based on our applied assumptions about the nature of that usage.

Disaggregating Usage

Usage can be disaggregated in a number of useful ways and assist with a more sophisticated understanding of the value of a journal subscription. Below are three models of usage disaggregation. Gathering this data in real-time might prove too labor-intensive, and perhaps infringe further on user privacy than most librarians are comfortable with. Instead, a sampling exercise could be performed a few times a year via surveys and other analyses.

Disaggregation Model 1: User Category  

With this disaggregation model a library would assign different valuations or weights to usage by different categories of users for each title, globally, individually or in some grouping.

Usage = U_f + U_g + U_s + U_o

In this example U_f = use by faculty; U_g = use by graduate students; U_s= Use by undergraduate students; U_o = Use by other constituencies. Depending on the library the categorization of users might be more or less complex. This model strives to be indicative of an approach the library may take.

To apply the model, a library may determine that usage by faculty for a particular resource is twice as important as usage by undergraduates and that usage by graduate students falls somewhat in between. For the purposes of valuing the use, they will exclude usage by other constituencies. (All of the different weights or coefficients assigned in this paper are for demonstration purposes only; they are not to be taken literally and presumably would vary by institutional type and other considerations.) In this example below, the use formula might be modified to look like this:

Usage = 1U_f +.75U_g +.50U_s + 0U_o

In this example each faculty usage would count as 1 use, each graduate student usage would count as .75 uses, and each undergraduate use would count as .50 uses. Use by other constituencies would be zeroed out. The net result for the purposes of calculation would be a decrease in the number of uses that get counted (the formula could just as easily result in an increase in the number of uses that get counted depending on the assumptions that get made).

In the example above a library might then estimate the value of a subscription-based resource with the calculation:

Value = (1U_f +.75U_g +.50U_s + 0U_o)/Cost

Here, value is equal to one times the number of faculty uses plus .75 times the number of graduate student uses plus .50 times the number of undergraduate student uses divided by the cost of the resource.

Disaggregation Model 2: Quality of Usage 

As we all know, sometimes general information about a topic is needed and a variety of information resources might meet the need, while sometimes a specific article is needed. We know too that sometimes there are information needs that can only be met by high-cost resources, yet also there are times when high-cost resources are used when a lower-cost resource would have performed equally well. This is where Disaggregation Model 2: Quality of Usage might be applied with the following formula:

Usage = U_n + U_c

Where Un= necessary use, that is, use of a specific article is necessary or class of resource is necessary; Uc= convenience use, that is, an article was used but a different information resource could also have met the need (e.g., lower division undergraduate papers).

As in Model 1, weights might be assigned to necessary uses and to convenience uses, so for a particular resource one might assign the following:

Usage = 1.5U_n + .25U_c

The resulting value estimation for this resource would then be:

Value  = (1.5U_n + .25U_c)/Cost

The value of this resource would be calculated as one-and-a-half times necessary uses (need for that specific article or class of article) plus .25 times convenience uses (another information resource could have met the need) divided by the cost of the resource.

Academic libraries provide a host of information resources for users and the annual costs for subscription-based resources vary greatly. Particularly with the advent of modern discovery systems, some of the usage that gets counted is essentially serendipitous – someone used an article because they came across it and it looked like it fit their need, not that they had a specific need for that particular article. There are many cases where a high-cost subscription resource meets an information resource need that could just as easily have been met by a lower-cost subscription resource with zero degradation in outcome.

Disaggregation Model 3: Time Requirement of Usage 

Subscriptions enable the convenience of immediate access and for generations libraries have worked to deliver our collections as quickly as possible. We have made assumptions about our need to compete in an environment of rising service expectations without necessarily assessing use cases to see what the true requirements are. If a substantial portion of the need for a high cost subscription resource is not immediate, a portion of the savings from canceling the resource can be reinvested to ensure that those with a true need for speed are not inconvenienced. The quality of interlibrary loan services has improved tremendously, particularly for article delivery. Next day delivery is commonplace, and it is not atypical for an article requested in the morning to be fulfilled by the afternoon. This is where Disaggregation Model 3: Time Requirement of Usage might be applied:

Usage = U_i + U_w

Where U_i = cases where there is an immediate need for access to an article; U_w = cases where user needs can be perfectly well met with a modest wait (e.g., Interlibrary Loan or asking author scenarios).

As with the other models, for Disaggregation Model 3 weights might be assigned here too so that for one resource a library might calculate usage as:

Usage = 1U_i + .50U_w

Here, each article for which immediate delivery was required would count as one, and each article for which a delay would be acceptable would count as .50 of a usage.

The ensuing value calculation would then be:

Value = 1U_i + .50U_w/Cost

There is definitely a service implication here and as always libraries would need to be mindful of degrading services. Moreover, it’s always important to remember opportunity costs, as dollars saved from investment in subscriptions may be repurposed for other activities central to the mission of the library, which might include a greater investment in interlibrary loan. What service levels are needed by which groups of patrons, and can we continue to provide premium level services in cases where a standard level of service is perfectly acceptable? A major Midwestern university recently announced it had prepared a list of titles for review for cancelation due to budgetary pressures. Of the list the chair of the history department noted “as I glance at the list I do see several items I don’t consider obscure or esoteric but I also have found interlibrary loan to be efficient at obtaining things we don’t have.”

Bringing It All Together

It should now be clear that the existing paradigm for expressing value in article subscription-based resources inadequately reflects variability in time, quality or category of user. A simple calculation of cost per use is inadequate for describing the nature of our patron’s actual use of the collections, and consequently not a sufficient tool for calculating value.

Old Model Valuation Formula:  Value = Cost/Use

I  have presented three models for disaggregating usage to offer a more refined calculation of usage that enables libraries to tailor-make assumptions to fit their particular users and then vary those assumptions based on the characteristics of each individual resource being evaluated. While each of the models has been presented individually, the complexity builds when one begins to pick factors from across the examples to build an even more robust model. Here is an example:

Usage = U_f + U_g +U_n + U_i

For this particular resource, a library might determine that the only usage factors it wanted to consider in valuing the resource were usage by faculty and graduate students, usage that was necessary and usage for which there was an immediate need.

The valuation formula for this resource would then look like this:

Usage = 1U_f + .75U_g +.9U_n + .95U_i

Value = Usage/Cost

Libraries will need to determine how to handle duplication; for example, an article usage will fall in multiple categories and therefore potentially be counted multiple times. A usage might be by a faculty member. It might also be a necessary usage and might also be needed immediately so that one usage could be counted three times. In a perfect world there would be a way to identify this overlap and correct for it by counting one use and apportioning that use to the different categories into which it might fall. Such work might be beyond our current data analysis capacities and ethical principles with respect to privacy, but it serves as the basis for imagining how far away the current standards are from reflecting true underlying value. And of course, while every day we are already making informal assumptions about journal usage– now we will have a more robust framework to use in structuring our thought processes.

So the next time a vendor tries to sell you on the value proposition that you should pay more because your cost per use is amazingly low you can tell them, “no, the dollars we invest in your subscription don’t generate enough quality usage compared to your competitor, so we are moving on, thank you very much!”

Thanks to Lisa Janicke Hinchliffe, Roger Schonfeld, Elizabeth Adelman, Jin Guo, Brenda Hazard, Kathryn Machin, Mary Van Ullen, Jill Dixon, James Galbraith, Irene Gashurov, Stephanie Hess, Caryl Ward and Mary Beth Kendrick for their substantive insights on earlier drafts.