ideaThis is an exciting time to be involved in publishing, librarianship, or scholarly communication generally.

“Exciting” is a nice, neutral term, isn’t it? There are so many different kinds of excitement: emotional anticipation, sensory stimulation, the eager expectation of a new experience, the dread of an incipient crisis, the terror of imminent death. These experiences may have little else in common, but I think they can pretty much all be said to preclude boredom.

Those of us working in scholarly communication have experienced many different kinds of “excitement” over the past 20 or so years, and I don’t think any of us expects things to become any less exciting anytime soon, for better or for worse.

One of the very cool manifestations of the exciting changes that continue to take place in our scholarly ecosystem is the emergence — sometimes gradual and deliberate, sometimes sudden and headlong — of new programs, products, and initiatives. Some of these emerge in response to changes already perceived in the ecosystem; some emerge in an attempt to create change where the status quo is deemed unsatisfactory. Some are designed to contribute new choices or functionalities to the system; some are intended to subvert or replace existing ones.

All of these programs, products, or initiatives, if they want to become going concerns, have one challenge in common: all must demonstrate initial feasibility and ongoing sustainability.

Most of us are familiar with the idea of “proof of concept.” A project or program demonstrates proof of concept when it shows that its theory can survive application in reality. In the library realm, where I work, we most often see proof of concept being demonstrated by means of pilot programs. A pilot program demonstrates proof of concept by showing in the real world that, yes, one’s library can develop, host, and maintain an open educational resource (OER); or establish a house-published journal and produce as least one issue; or provide a home for a digital archive; or establish an award for outstanding student service. By doing something like that once, we demonstrate that it can be done. That’s proof of concept.

Here I’d like to suggest two additional dimensions of proof that are necessary when establishing proof of concept for ongoing projects or initiatives: proof of program, and proof of scale.

Proof of program would mean demonstrating that the initiative one has created is sustainable over time: having developed the OER and made it available to the world, can we keep it going indefinitely? Having published an initial issue of a journal, or an initial volume in a book series, can we continue generating new content, keeping up with the editorial work, and curating the content indefinitely? Having created the digital archive and populated it with an initial batch of content, can we continue adding content to it and providing effective curation? Having established an award for student service, can we keep funding and administering that award from year to year? By keeping such initiatives going over time, we demonstrate that the idea was not just a one-off success. That’s what I would call “proof of program.”

Proof of scale would mean demonstrating that the program can be maintained not only indefinitely (or for whatever length of time is desired), but also that it can be maintained at whatever volume of output is needed or desired: the desired number of issues per year (and of articles per issue), the number of books created per year, the number of awards granted per year, etc. While it’s one thing to demonstrate the possibility of creating and maintaining an OER or a journal over time, it’s a very different thing to show that you can create and maintain as many of them as you’d like to, or keep the content flowing in the amounts desired–particularly if your journal succeeds to such a point that it starts attracting many more submissions than it was originally designed to handle. Does issue #16 of your locally-produced quarterly journal have as many articles (of comparable quality) as issue #1 did? If so, that’s proof of scale.

In the context of academic libraries, though, these three dimensions of “proof” interact in at least one interesting way: when one library demonstrates that a project can successfully be undertaken (proof of concept) and that it can successfully be maintained over time (proof of program), other libraries may follow suit. The original library might have wanted to establish five journals but only succeeded at creating and maintaining one; however, if other libraries see the first library’s initial success and undertake the same type of project (perhaps using templates, platforms, or programming invented by the first one), the resulting impact on the ecosystem will be the same as if the first library had achieved its scale goal. That library may not prove that it is able to carry out the project to scale on its own, but it may still be proved that the project itself is scalable. In other words, if we’re thinking about scale at the institutional level, it’s measured locally; if we’re talking about it in the context of scholarly publishing as a whole, then scalability may be demonstrated across multiple institutions — especially if the programs in question involve open access or other kinds of freely-available content and services that behave cooperatively rather than competitively, and that therefore can be experienced by readers as a suite of offerings.

So what might these three different concepts of “proof” look like in the real world? Here are some examples:

The journal PLOS ONE has demonstrated proof of concept, proof of program, and proof of scale: it has successfully published not only an initial batch of articles, but has reiterated that success over several years, and has done so at a very large and growing scale. (While it’s true that PLOS ONE’s size may be undergoing something of a correction at the moment, that doesn’t change the fact that it has demonstrated the ability to publish at roughly its intended scale over time, and to grow year on year.)

Luminos, a recently-inaugurated open-access monograph program of the University of California Press (previously discussed here in the Kitchen), has published its first batch of three scholarly books, and has announced the imminent publication of seven more. By publishing the first three, Luminos has achieved proof of concept for its unique business model; when it releases the next seven (and as it shows its ability to sustain access to the previous titles), it will have gone some distance towards demonstrating proof of program; to the degree it is able keep publishing new content at the rate desired, it will demonstrate proof of scale. Similarly, Knowledge Unlatched demonstrated proof of concept with the release of its initial batch of 28 books in 2014. Its next batch is scheduled for release in March 2016; if that happens, and especially if that batch is followed by another successful release, Knowledge Unlatched will have demonstrated both proof of concept and proof of program. Proof of scale will be indicated by the ongoing frequency of those releases and the number of books in each batch. (The fact that 80 titles are projected for the second batch is promising in that regard.)

The Open Access Network (OAN) is a highly ambitious program that solicits subventions from research and academic institutions (not just their libraries) in support of open access journal publishing projects by scholarly and learned societies and university presses. Under this program, academic institutions will pay into a central fund, to which societies and presses would apply for grants to allow them to develop freely-available scholarly products. The OAN is quite new and is still in the process of fundraising, and thus has yet to demonstrate proof of concept. Interestingly, in this case concept, program, and scale are all baked together: ongoing programming and global (or at least national) scale are both intrinsic to the OAN’s fundamental concept, making it an unusually ambitious project.

Our current scholarly communication system is in a pretty dynamic state right now (though it’s not chaotic, at least not yet), and its legacy structures are under constant and maybe even increasing pressure from people and organizations who, in varying degrees and with varying degrees of friendliness to those structures, would like to see them change. So we can expect to see a pretty constant stream of such new initiatives arising over the coming years. It will be continue to be interesting to see how many of those initiatives are able to demonstrate proof of concept, proof of program, and proof of scale—and how each of them does so.

Rick Anderson

Rick Anderson

Rick Anderson is University Librarian at Brigham Young University. He has worked previously as a bibliographer for YBP, Inc., as Head Acquisitions Librarian for the University of North Carolina, Greensboro, as Director of Resource Acquisition at the University of Nevada, Reno, and as Associate Dean for Collections & Scholarly Communication at the University of Utah.

Discussion

20 Thoughts on "Proof of Concept, Proof of Program, and Proof of Scale in Scholarly Communication"

Unless one is feeding at the public trough one has the fourth leg of the stool (to mix metaphors) and that is can it pay for itself? In short, does anyone want it! Will publicly supported institutions continue to support cost centers such as book publishing when there are other things more central to the university’s mission?

The problem with the phrase “feeding at the public trough” is that it implies public employees are taking stuff out and not giving anything back. It would be more accurate (and less obnoxious) to say “working on the municipal farm.”

That said, your question about whether publicly-funded institutions will continue to support publishing ventures such as the ones described above is, obviously, centrally important. And the answer should be “Yes, if those ventures are in harmony with the institution’s mission and if supporting those ventures is the best way to further that mission.” Those are both big “ifs,” of course.

I don’t truly think you can you “proof of program” until the proof of concept demonstrates some degree of financial sustainability.

As you’ve told us before, Rick, even if those venture are harmonious with your institution’s mission and supporting those ventures is the best way to further that mission, if you don’t have the money, you don’t have the money. Practicality trumps philosophy almost every time.

Absolutely. Every institution has to set priorities, and when you have to set priorities there are always winners and losers. This is a real challenge for scholcomm initiatives, which are often very exciting to the library but (rightly or wrongly) much less exciting to the university administration.

Harvey’s comment touches on the thought that I had reading the post – “what about proof of market?” Perhaps you were taking that as read, Rick, i.e. you wouldn’t get to proof of concept if you hadn’t undertaken analyzed your market, the demand within it and (depending on context, your points about academic libraries being well taken) the business models by which your product might successfully function in that market. Perhaps “proof of market” merits a prequel posting as I’m interested in what it would entail outside of a straightforward commercial context, and how much it happens in the “municipal farm” context (or do initiatives get taken forward on the assumption that there is a market? and is that assumption usually correct such that proof of market might be overkill?)

Excellent question, Charlie. And you raise another important question in connection with it: when you work on the municipal farm rather than on an independently-owned or publicly-traded farm, is there an enhanced risk that the products you bring to market will be things you believe people ought to want rather than things that they demonstrably do want?

I think you’re right that there’s another posting in that question.

The issues of scale and sustainability remain huge questions for so many of these efforts. We have a publishing system that has endured and evolved for 350 years and that handles an enormous number of articles annually, yet many wish to replace it overnight with new ventures that may not last very long and that may not work all that well if they had to cover all of the literature.

We have new OA journals that are run on a volunteer basis and that charge authors no fee, and these have had some success covering limited areas of the literature and publishing relatively small numbers of articles. It’s unclear how they might scale up. Volunteer librarians and researchers can cover the work to publish tens of articles per year, but what happens when their submission rates rise into the thousands? Suddenly running the journal becomes a full-time job, and that requires payment for employees. Similarly, electronic management systems need to scale up, again requiring development and maintenance costs. Is a university going to be willing to massively scale up their contribution to covering those costs? We know that funding agencies are great at funding the creation of new entities, but notoriously bad at funding the maintenance of existing entities. What happens when the grant money runs out?

Similarly, we see privatization of peer review services that are often based on a small group of ex-researchers and their contacts within the field in which they did their degrees. Are these companies capable of expanding their reach to all realms of academic research? Can a small company effectively provide peer review services for string theory, pediatric psychology, Drosophila neurobiology, social work, cardiac surgery, Australian law and eighteenth century French poetry?

Seeing the publishers above put out a second round of books is indeed an encouraging sign, but is there a tipping point where most would feel comfortable putting all of academia’s eggs into their baskets? Is that point 5 years of proven success? 10 years? 20 years? 100?

Those are all good and vital questions, and I confess that they trouble me as well to some degree. Though when it comes to initiatives like these, I’m not sure we need to define success in terms of attracting all of academia’s eggs. I’m pretty sure that the future of scholarly communication and publishing will continue to feature lots of different baskets and that academia will distribute its eggs among them. I’m aware that there are some who would like there to be only one basket (or at least for all the baskets to be the same color), but I can’t really see that happening, for a variety of reasons.

As with most things, moderation is usually the best path forward. To use a tired metaphor, a diverse ecosystem is a healthy ecosystem, able to adapt as different selective pressures come to bear. We’re often dealing with extremist advocates, or opportunists looking to promote their own product as the one true way, and both of these are problematic for a system that should serve a very diverse set of needs as well as one that needs to last for the long term.

That doesn’t mean that these developments are without value or are not welcome in that diverse ecosystem–really, the more experimentation the better. It becomes problematic though when such unproven properties are suggested (or even worse legislated) as a complete replacement for everything else.

Another way of asking David’s question is to ask whether university presses, even collectively, could take over all the publications of the commercial giants like Elsevier, Taylor & Francis, Wiley, etc. I believe, historically, that the reason STM publishing, which was originally handled by university presses and scholarly societies, was eventually dominated by these giant enterprises was that universities and societies neither had the funds available nor the will to dedicate them to expanding the scale of their publishing operations to this extent. And there is a very real question today whether, if these commercial giants decided to exit the business, whether the non-profits could cope with this scale of publishing. I think not.

I think Sandy is asking this backward. The key point is that the U. press world has not succeeded. With every day more and more of scholarly publishing moves away from them. The U. press world has not kept pace with the overall marketplace. OUP may be an exception to this. Other outstanding presses (and there are wonderful editorial programs at many presses) are not much larger today than they were 20 years ago, despite the fact that scholarly communications has been marked by considerable growth.

I’d push at that “relatively small numbers of articles” a bit. Relative to what? By my count, no-fee OA journals (excluding problematic ones) published 206,588 articles in 2014. That won’t replace the whole scholarly publishing system (and I don’t envision 100% replacement anyway), but it’s non-trivial. (Within the humanities and social sciences, where no-fee publishing is strongest, 75,000-odd articles in no-fee journals is far from trivial.)

I was thinking “relatively small numbers of articles” in terms of numbers of articles per journal. If we’re talking 6,400 journal publishing 200,000 articles per year, that’s roughly 31 articles per journal per year, 2.6 articles per month. That’s a lot smaller than most of the journals I deal with. And I have regularly seen statements about how most OA journals are free for everyone, so all journals should eventually become that way. My point is that it may be feasible for part-time people to deal with a handful of articles in their spare time, but once you get the typical article flow for a major journal, its a full-time job for multiple people. Looking at the opposite extreme, from PLOS’ 2013 990 statement, they spent close to $7.5M solely on outsourced editorial services, which I don’t think is feasible for most libraries.

Thanks for the clarification; with that clarification, I pretty much agree with your point. Most volunteer OA journals are smallish, and that makes sense (but then, at least in the humanities, smallish journals are frequently appropriate). There are other non-APC funding mechanisms (e.g., SciELO, which includes some fairly sizable journals), but I’ve said before that it seems improbable that entirely-voluntary efforts are going to work well in journals publishing hundreds or thousands of articles a year.

The problem I see with defining what proof of program and proof of scale are has to do with how much subsidy a parent university is willing to provide to start, sustain, and grow publishing operations. There is quite a degree of variation now among the subsidy amounts universities provide to their presses, so how does one answer the questions about sustainability and scale without knowing what a university is willing to invest? Most of these programs do not make a profit, so the choice has to do with how much by way of losses over time a university is willing to absorb. If a monograph series loses $5,000 a year, is that sustainable? If it loses $100,000 a year, is it sustainable? I don’t think there is any “one size fits all” answer to such questions, any more than there is a single answer to how much universities are willing to invest in their libraries. If you require a program to make a profit, then almost none of these programs could be considered sustainable or scalable. PLOS ONE is an obvious anomaly here.

If you require a program to make a profit, then academic libraries aren’t sustainable.

Thanks for a thought-provoking post, which raises a number of excellent points. Responses here seem to be primarily from an STM-journals perspective so let me focus on monographs. In this particular case, while the current system may have been in place for some time, the fact is that it isn’t really working for anyone. As presses, we lose money on most monographs we publish; libraries can no longer afford to buy the majority of our output; authors are ill-served by a model that consigns their scholarship to the few hundred western libraries that can afford them; and the scholarly record itself is defined by market-based decisions about what gets published, rather than scholarly judgment.

Like other OA monograph initiatives, Luminos is a serious attempt to grapple with these problems. It does indeed need to pay for itself, but that’s all it needs to do from a financial perspective. (All of the revenues created by Luminos stay within Luminos to fund publication costs and author sponsorship.) We’re tracking revenue and costs very carefully but given the significant cultural shift this model represents for these disciplines, we’re also closely following other metrics. Will authors submit, and what are their reservations? Will these books “count” for promotion and tenure? How many libraries will be prepared to support us? What is the real audience for this scholarship without paywall constraints? Can we resolve the very real issues of licensing models and third party rights? So far, things are looking good in all of these areas but it is still very early. As I’ve said all along, Luminos will not answer all of the problems of the monograph (and like others here, I am firmly in favor of a diverse ecosystem), but for monographs at least, we should welcome all attempts to fix a dysfunctional system.

I think that we have to be rather careful when asserting as an accepted fact ‘the current system for the publication of monographs isn’t really working for anyone’. The actual global situation is a lot more nuanced than that, as I am sure Alison would agree. Otherwise, why would the (greatly exaggerated) ‘death of the printed monograph’ have been a subject, on the whole, for melancholic reflection, rather than an outcome to be celebrated? I have always tended personally to Rick Anderson’s pragmatic view, that the major reason ultimately for the curious lack of traction of Open Access amongst significant numbers of scholars, especially those in the arts and social sciences, is that actually the current systems work well enough for 80% of western tenured faculty in 80% of research institutions in 80% of cases. That’s not to say, of course, that there are not major improvements that could and must be made in the search for long-run sustainability and improved dissemination (as clearly U Cal P are trying to do with Luminos), but thus far the most impactful such improvement has been the very-short-run printing option made possible by the first wave of the digital transition, and greatly facilitated by the growth of Amazon and the massive developments in bibliographic search. To an often under-articulated degree, this revalorisation of the backlist saved the monograph.The growth of various e-monograph propositions subsequently has not (yet), I would argue, been anything like such an influential (and benign) or transformative initiative, although of course the best may be yet to come…

I am always genuinely puzzled by the lack of attention paid on the SK and elsewhere to the huge commercial monographic programmes of (say) Routledge or Palgrave (very active in the Open Access sphere) or Bloomsbury: T&F Routledge have just spent an eight-figure sum to acquire the 14 000 monographic contracts of the long-standing UK imprint Ashgate, a major player in (e.g.) Reformation Studies, suggesting that for some publishers at least (including, notably, both Oxford and Cambridge) monographs remain a healthily viable financial proposition. Monographic output from the UK’s Big Four presses in this context (OUP, CUP, Routledge, Palgrave) more than doubled in the first decade of this century, as readers of the recent Crossick Report for the UK Higher Education Funding Council will have seen. Quality control is sometimes raised as an issue: whilst refereeing at both OUP and CUP remains generally very tough-minded indeed, with the faculty Delegate or Syndic role still absolutely central to the publishing decisions of the world’s two biggest university presses, it is also very striking to look at the sheer numbers of those monographs issued by commercial imprints (like Routledge and Palgrave and Bloomsbury) and thought worthy of submission by their authors to the most recent UK Research Excellence Framework exercise, and indeed purchased by major research libraries around the world (including those in the USA).

This is all by way of saying that as things stand there is more than one way to approach the monographic issue, and whilst clearly variants of the AAUP experience provide the dominant context for understanding in North America, that experience is by no means the only one, and nor, importantly, is it the only one for large numbers of US-based faculty, any more than faculty on the other side of the Atlantic restrict themselves to the traditional European-based publication possibilities. The book exhibits at AHA or APSA or MLA, and the nationalities of those both exhibiting and attending, make this exuberant internationalism abundantly clear. In this respect, at least, the monographic ecosystem already manifests a healthy diversity, but self-evidently we need still more.

Richard, you raise a number of excellent points here, most importantly the one about geographical diversity. The ecosystem in the UK in particular appears to be much healthier for monographs with greater diversity and output than say here in the US. That said, I think it’s also clear that there is more than room for improvement. Innovative and new forms of scholarship as well as junior scholars (where the market may be smaller and the risks higher) find it much harder to get published than established topics and authors. This kind of work does not easily find its way onto the lists of the publishers you cite, and even university presses are finding it hard to support this scholarship with a traditional market-based funding approach. The long-term implications of that are ones that should concern us all – as John Willinsky noted “The limited economic viability of the monograph puts the university press in a position of restricting, in effect, what is studied and, as such, acts as a check on academic freedom and scholarly judgment about what historical work is needed.”

Moreover, many of the current publishers cited (Ashgate is a great example) rely heavily – although not exclusively – on a high-priced hardback model and, while that may do a better job of covering the costs of publication for publishers, there are fairly obvious ways in which this doesn’t do such a great job of serving the interests of authors, libraries and scholarship more widely.

So while I would certainly agree that there is much of value in the current system that we need to retain, I think there is room for significant improvement for those of us who are truly committed to the important missions of our institutions.

Comments are closed.