[Editor’s note: This essay first appeared in Choice. Joe wishes to thank Tom Radko, the editor of Choice (and Joe’s former colleague) for permission to reprint it here. The essay itself is based upon a presentation that Joe delivered to the ICOLC conference in Albany, NY on April 27, 2015. The slides for that session can be found here. The slides were in turn summarized on the Scholarly Kitchen. For information on ICOLC—the International Coalition of Library Consortia—see http://icolc.net.]

I wish to describe a situation that I often come across and then try to explain how it came about. What are its implications for scholarly communications and library consortia in particular, and can we project into the future to think about how this situation could evolve?

So here is the situation. Please note that throughout this discussion, I am presenting the case as professional societies see it. I frequently am contacted by these societies to help them with their publishing programs. Most of these societies are in the STM area, but humanists occasionally make that phone call, too. Virtually all of these conversations are about journals, not books or databases. I should add that I am not the only person who gets these calls. There is a whole little industry of consultants who work with professional societies. This is because not-for-profit professional societies often lack experienced business people, so the leadership looks outside when problems arise.

The typical phone call goes like this:

We are a scientific society that has been around for many years. We are the leading society in our area, though most of our members belong to other societies as well. We publish 3 journals, which are among the best in the field (as measured by Impact Factor). These journals used to throw off a healthy profit, but over the past several years the number of subscribers, both individual and institutional, has been dropping. This is problematic for us because we historically used the journals surplus to fund other activities. For example, we use the surplus to provide subsidies to graduate students to attend our annual conference. The journals are still modestly profitable, but when we look at the trends, we can see that in time they will be losing money. We don’t have the resources to subsidize the journals. What should we do?

Now let me back up a bit to provide some caveats and disclosures. First, I am not a librarian. Second, I am not a publisher, though I was trained as one many years ago. I work with organizations on their strategies. At this time most of these organizations are in the not-for-profit area, though I continue to maintain some commercial clients. I don’t have a “philosophy” of publishing or of scholarly communications, and I am skeptical about people who do. My own preference is to study a situation to try to determine how it got where it is and then to come up with ideas about where it can go. This is not an outlook that will help anybody change the world, but it is helpful for people who want to change an organization.

Now we turn back to our professional society. How do we advise this group?

Depending on the ingenuity of the consultant, this could be a short conversation or a long one, a small project or a big one. Let’s imagine a project that goes through every possible step, though in practice almost none ever do.

So the first step is analysis. Why are the individual subscriptions dropping? Here we find that the culprits are libraries and digital technology. This may be a familiar story. A society will recall that 25 years ago membership included a free copy of the society’s flagship journal. It was delivered in print. Many people joined the society to get the free journal and were pleased to receive it by mail every quarter or every month. But then the Internet came along and the society invested in a Web site. Institutional subscriptions began to rise, and that was a very good thing. But the subscribing libraries enabled remote access to the journal in electronic form, so the same researcher could get access to the material from the comfort of his or her home. Many researchers then began to cancel their memberships and personal subscriptions.

There are some important implications in this situation. First, the new library-driven digital service provides increased value for the end-user. That’s a good thing. It also makes the library more valuable to the end-user, and all libraries have that as a goal. Simultaneously it subtracts value from the publisher. Not surprisingly, publishers don’t like this. From the publisher’s point of view, the revenue lost from membership fees and individual subscriptions had to be made up somewhere, and societies turned to libraries to offset the lost revenue. So the burden on library budgets began to rise. This is not the only reason publishers’ prices rose, but it is a major one. In effect, as libraries became more valuable to their users, they wound up paying a handsome price for it.

The growth in the value of institutional subscriptions was a bright spot for our professional society, but it was short-lived. When everybody began to use the Internet, when the virtual world got more and more crowded, the society began to see that its institutional subscriptions started to fall off. This is despite the fact that the journals were able to maintain their Impact Factors. And this is precisely what precipitated the society’s current crisis, the fall-off in library subscriptions. Here there are many culprits. Flat library budgets are the leading villain, but the Big Deals of the large publishers also play a major role, as they take up a huge amount of a library’s budget. Now, this is an area where librarians know a great deal more about the situation than I do, but let me present the situation as societies see it. Society publishers often complain that they cannot even communicate with libraries and library consortia because everyone is too busy negotiating arrangements with the likes of Elsevier and Springer Nature. This is a common complaint among society publishers, and if it is unfounded, there needs to be a communications program to correct this.

Faced with these challenges in the marketplace, many societies then decide that they have to become better publishers. They begin to attend all the trade shows; they bring in a horde of consultants; and the outcome is that you can then see these societies upping their game as journals publishers. To focus on production and workflow, they make arrangements with companies like Allen Press and Cadmus. If they don’t think they have enough marketing clout, they may sign an agreement with Accucoms or PCG. A particularly important decision concerns the software hosting platform for delivering journals online. RFPs go out to firms like HighWire and Atypon. All together, this process takes about 2 years. At the end of it, the society can rightly claim that they have raised the bar: they are indeed better publishers now because of the strategic partnerships they have built.

There is another implication to this, however, and that is the hollowing-out effect. All of these partnerships involve outsourcing; with every new agreement, the society does less and less on its own. It raises the important question of how much a society can outsource and still retain its identity.

Unfortunately, our professional society eventually comes to the conclusion that, for all the effort that was put into it, the partnership and outsourcing plan doesn’t work. Library subscriptions continue to drop. Ten years ago, it may have been a different story, but now simply having good vendors is not enough. I call the situation ten years ago “the HighWire era,” when working with HighWire would almost guarantee the success of a society publisher. But those days are gone. HighWire has lost its centrality because fewer societies can thrive as an independent publisher with a suite of vendor relationships. What societies perceive is that they need to outsource even more; they need some scale to assist in marketing. And the reason for this is that they believe that they are too small to get a library’s attention.

The next step for many societies, especially smaller ones and many in the humanities, is to explore the world of the American university presses. This is a smart move. The American presses (I am specifically excluding Oxford and Cambridge in this argument) collectively publish about 200 journals. About half of the presses publish journals. The largest journal portfolios are found at Johns Hopkins, Chicago, Duke, MIT, and California. All of the presses have a collection of relationships in place with libraries, subscription agents, wholesalers, and vendors of all categories. A professional society can make an arrangement with a university press and immediately see its costs drop and its marketing program intensified. In some instances arrangements with university presses deliver a surplus, sometimes called a royalty, to the societies whose journals they publish. So this could be a good move for the society, and many make it.

Up until a few years ago, I believed that advising a society to work with a university press was about the best advice I could offer except for those societies that were working in STM fields where access to markets in the developing economies was critical. The arguments in favor of a university press are very good. They provide scale, they lend their imprimatur to the society (who wouldn’t want to be published by The University of Chicago Press?), they provide a soup-to-nuts set of services for everything from production to archiving solutions with Portico, and they keep the revenues in the not-for-profit sector. I no longer believe this, however, except in some special cases. The turning point for me was about five years ago, when the American Anthropological Association moved its AnthroSource portal from The University of California Press over to John Wiley. It became clear at that time that the largest publishers were now targeting the journals of the American university press community. Since then we have seen several other instances of poaching at, for example, Chicago and MIT, and all of the journals directors in the university press world know that they are dealing with very stiff competition.

So why would a professional society choose Wiley over California or Elsevier over Chicago? The primary answer is global distribution and the revenue that that distribution provides. The key aspect of that distribution system is that the larger publishers more easily can get the attention of libraries and library consortia. Let me repeat a point I made earlier: If it is not true that small publishers cannot get the attention of library consortia, then there is a profound communications problem that must be addressed.

It hardly comes as a surprise, however, that many societies have reservations about working with large commercial organizations. It can actually be highly amusing to attend a Board meeting of a professional society and listen in on a discussion about working with Elsevier. One would think that there is indeed a devil and the devil speaks Dutch. Some of the remarks about Elsevier, and to a lesser extent about Wiley and Springer Nature, are so over the top that there is no way to respond to them. Every society has some members with deep reservations about commercial publishers. So how does the society deal with this situation, where the American university presses don’t seem to be robust enough and Elsevier and its ilk are unacceptable to some of the members?

Here our intrepid advisor, who has been retained first to do an analysis, second to assist in assembling a stronger set of vendors, third to broker an arrangement with an American university press — our advisor now counsels that the society seek a new arrangement, this one with a larger entity that happens to also be in the not-for-profit area. The fact is that there are not too many of them; as a practical matter the society is pointed to the university presses at Cambridge and Oxford.

Can Cambridge and OUP do the job? Yes, they can. They both have large journals programs (between the two of them they publish around 600-800 journals, a number that is rising). They both have programs in the humanities and the sciences. They both have global footprints. They both have sufficient scale to have competitive technology and to get good pricing from other vendors. And they both are determined to remain competitive in journals publishing. These are not small companies. At roughly $350 million a year, Cambridge is in the ballpark of Sage. OUP is a billion-dollar company. But most importantly, both of these organizations are owned by universities. They are not-for-profit and are aligned with most professional societies in being mission-based. I have personally been very impressed with the competitive zeal that these companies have brought to the journals business in the past couple years.

So it looks as though our society’s problems may have been solved. If the core problem was declining library subscriptions, an arrangement with either OUP or Cambridge can fix that. They are big enough to get the attention of libraries and library consortia, thus they can generate the revenue that the society requires.

For some societies, however, this may not be enough. In some instances OUP and Cambridge find themselves competing with the commercial firms. Occasionally a society desires that an auction be set up. I have run some of these and I will tell you that the bidding can be dizzying. A big auction can include OUP and Cambridge on the not-for-profit side and Elsevier, Wiley, Springer, Taylor & Francis, and Sage on the commercial side. There have been instances where the large commercial firms make very rich offers to professional societies that are hard to pass up. An example of this is the American Geophysical Society, which went from being a very good independent publisher to signing an agreement with Wiley. A great deal of money changed hands. When a large company wants an agreement with a particular society, they have the ability to outgun the largest of the not-for-profits.

So now we see where this long process was headed. It began when a society reviewed its own accounts and saw that its library subscriptions were declining, and it ended with an agreement with one of the largest commercial firms, which promptly dropped the new journals into a Big Deal or other bundles. Often these arrangements entail stiff price increases to libraries, which in turn yield greater revenue for both the society and the large publisher. The society also gets the benefit of state-of-the-art technology, broader dissemination of the research they publish, and a powerful marketing partner.

The libraries, however, may have found that the balance of power has shifted uncomfortably, as it is very hard to say no to Elsevier and Wiley. In this scenario, it could be said that the society gains through greater revenue, the commercial partner gains through revenue and increasing marketing influence, authors gain through broader dissemination, readers gain through integrated platforms and the near-ubiquity of the content from the large publishers. Libraries, of course, are now footing the bill for the entire enterprise. So perhaps libraries can be said to be the losers in this situation.

Let’s make sure we understand what has happened here. A professional society wants to get its journals into libraries, but finds the gates are closed. So the society ends up making a deal with a large publisher, which serves as the gatekeeper to the library budget. You might ask, What if we gave priority to small publishers? Would that reduce the market clout of the big commercial firms? In other words, libraries have been complicit in the creation of the marketing environment we see today.

If someone wanted to change this situation–for example, to make it friendlier to libraries–where would you begin? Here are some strategies:

*Libraries can organize themselves into consortia to give them greater influence with publishers.

*Support open access publishing.

*Develop publishing centers within libraries.

*Build institutional repositories to create an alternate source for materials.

Let’s look at these in turn.

First, as for library consortia, I know less about them than I would like to; I can only report how publishers see them operating in the marketplace. For small publishers it is harder to sell things to consortia than it is to individual libraries. This then stimulates a small publisher to talk to a larger publisher about a service arrangement. Consortia, in other words, accelerate the process of consolidation among publishers. So we end up with a situation that looks a bit like heavyweight wrestling, with huge consortia on one side battling huge publishers on the other. What the large publishers have perceived is that the small publishers need them now more than ever. A cynic might say that the largest publishers have coopted the consortia.

Open access, on the other hand, is a very different animal. I find it hard to understand why so many librarians support open access publishing because its practical effect is to make libraries less central to scholarly communications. I can make very good arguments for OA publishing, but the benefit seems to me to flow to the publishers, not the authors or libraries. Gold OA provides an additive revenue stream for established publishers, making it possible to monetize many articles that may otherwise have been rejected. The value chain for Gold OA involves funding agencies, authors, and publishers. Librarians play a small role in managing APCs on behalf of their institutions. The larger publishers used to fear OA, but the Gold model has made them into enthusiasts. I don’t see how OA is likely to lessen the claims that the largest publishers have on library budgets. Essentially, these publishers have coopted the Gold OA marketplace, which has boosted their revenues.

Green OA, on the other hand, is more complicated. I will get to that in a minute.

Library publishing is distinct from OA publishing, though virtually all library publishing programs I am aware of are in fact OA. The principal behind library publishing is that one way to diminish the influence of publishers is to compete with them. That’s a good idea. Library publishing, however, then takes a turn that works to its disadvantage, and that is that it gives priority, if not an exclusive relationship, to the faculty of the library’s parent institution. In effect, library publishing is a house shop, the digital equivalent of the old campus print shop. Professor Jones in the department of anthropology writes an article and uploads it to a library program, which may be organized into journals or may be a “megajournal,” that is, encompassing several or all fields. The library publisher takes the manuscript, processes it, and then mounts it on a Web server. There is absolutely nothing wrong with this. It does not, however, compete with formally published material from established publishers.

Here’s why. While library publishing is a house operation, focusing primarily on the output of its own faculty, traditional publishing seeks authors wherever the very best ones can be found. Traditional publishing, in other words, is editorially focused; library publishing is production-focused. Two good things, but two different things.

It’s instructive to compare university presses to library publishing. University presses publish only a small number of books from their own faculty; typically under 10% of a press’s list consists of books by authors from the parent institution. Presses do this deliberately, as they don’t want to be viewed as a house publishing arm. They then have the material peer-reviewed by experts from a multitude of institutions. The reason they do this is that traditional publishing is primarily about certification, not about dissemination. Library publishing, with its focus on its own faculty, mostly bypasses the certification issue. So we have two rival systems serving different ends. I repeat: there is nothing wrong with this; they can happily coexist. But what library publishing does not do is undermine the influence of the largest commercial publishers.

Institutional repositories represent a truly new platform for scholarly communications. I don’t know of any large institution that does not have an IR. My understanding is that they have not lived up to their promise, at least if one looks at the paper that SPARC commissioned from Raym Crow many years ago. (You can find Raym’s piece here.) Raym argued that IRs could become alternatives to digital collections from established publishers. That hasn’t happened. For the most part IRs have become the place to deposit unconventional documents. Here again there is nothing wrong with this. It simply is not a competitive challenge to the established publishers.

Could Green OA change that? This will be hard, but it’s not impossible. The U.S. government has a Green policy, so the article have to be made available somewhere. Libraries are one candidate. The publishers have organized CHORUS precisely to retain control of their publications even if they are OA. PubMed Central is likely to grow; whether or not it morphs into PubFed Central, as many wiseacres call it, which is the publishers’ worst nightmare, is yet to be seen.

I suspect that IRs are not likely to be significant players in Green OA for technological reasons. Even now the software for most IRs is licensed from third parties, Digital Commons in particular. If Green OA became a bigger thing, as it might, then it is likely that commercial centralized services will be developed that will have the benefits of scale and access to capital markets for development. Perhaps the emergent model will be that of some of the portfolio companies of Von Holtzbrinck’s Digital Science, ReadCube and FigShare in particular. Digital Science is investing money in companies that outsource library functions; in effect, they see a trend for libraries to be hollowed out just as independent publishers have been. My crystal ball tells me that IRs will be of interest primarily on a local level and mostly for undergraduate documents.

So is this a gloomy outlook or a cheerful one? It depends. On the cheerful side, we would cite the following:

• The current situation lends itself to scale, consolidation, and operating efficiencies
• The current arrangement also reduces transaction costs (fewer customers) and enables significant investment in technology
• The current arrangement has driven down the cost per article (even while it has substantially increased total costs)
• More material is available to researchers than ever before (we should not overlook this)

The gloomy side of this is as follows:

• Publishers exploit their size and drive up prices
• Large established publishers effectively “own” library budgets and thus block attempts to reach out to new suppliers (because there is no money left)
• The current system lends itself to further consolidation and more clout for the largest publishers
• The marketplace dominance of the largest publishers effectively reduces competition and suppresses innovation

If you take the gloomy view, what is a library’s options? The most important thing is to avoid the devil’s bargain in which an organization seeks reductions in transaction and administrative costs and pays for them by inadvertently granting a company with a monopoly gateway that can be used to influence the structure of the marketplace. On the horizon we have the GOBI service, which is a monopoly in the making. Today it is an excellent tool for reducing costs, tomorrow it could become a dominant gatekeeper that could reshape much of the value chain for scholarly communications. Libraries may wish to have a plan to increase, not decrease, the number of suppliers and force them to compete with each other, even as libraries make a point of keeping all these suppliers alive by apportioning their budgets across the entire field.

But most importantly libraries and consortia could give first priority to the smallest publishers. There is a cost to this in having to apply more administrative attention, but it is an investment in maintaining control of the marketplace. My example of the small society publisher should be taken to heart. If that publisher could find an audience with libraries and consortia, then there would be no reason for it to cut a deal with John Wiley or Elsevier. Libraries could change the structure of the business of scholarly communications if they wanted to, but first they must understand how they have been complicit in establishing the very system that seems to penalize them so much.