Library card catalogue
Library card catalogue (via. JISC collections)

A report detailing an experiment in providing open online access to scholarly monographs was released last Tuesday in celebration of Open Access Week.  The results? Providing free access to online books had no effect on sales (positive or negative), nor did it have any effect on citations. Free access did increase online usage, however.

I wrote about this study almost three years ago to date as the Joint Information Systems Committee (JISC) announced their financial support of this study. At that time, I argued that the study was methodologically flawed, would yield null results on sales and lack generalizability.

Did we learn anything in the past three years? The following post is reposted from Oct 27, 2010.

[Correction: This post confuses the Dutch OAPEN-NL study with the British OAPEN-UK and the broader European OAPEN initiative. Details of the differences are described by Ronald Snijder in the Comments section below. The author (Phil Davis) regrets these errors.]

OAPEN — Open Access Book Experiment in Humanities, Social Sciences

The Joint Information Systems Committee (JISC) in the United Kingdom has put out an invitation for publishers of monographs in the social sciences and humanities to participate in an open access experiment called OAPEN-UK.  As described:

The aim of OAPEN-UK is to experiment with scholarly monographs in the humanities and social sciences to find out if open access as a model is feasible, and what impacts open access scholarly monographs have on print and e-book sales, reach and readership.

The study comes with £250,000 (almost US $400,000) of support from JISC-collections to fund the experiment.

Like the open access experiment conducted on monographs published by Amsterdam University Press (AUP), books will be assigned randomly to either the experimental (open access) group or the control group.  To ensure equal allocation, each publisher is asked to submit pairs of titles that are similar to each other in many respects (age, subject, e-book availability).

Matching subjects is a common experimental design in the social sciences that allows the researcher to control methodologically for variation that is known to exist between individuals, or in this case, books.  Instead of testing for differences between each group of books (e.g. A, B, C vs. D, E, and F), the matching approach analyzes the differences between each pair of books (e.g. A-D, B-E, C-F). Using the latter approach often results in a more sensitive statistical analysis.  But I’m speculating here, because the methodology and statistical analysis are not spelled out in the invitation to tender.

The explanatory variables under investigation aren’t any clearer. While the study asks, “what impacts open access scholarly monographs have on print and e-book sales, reach and readership,” nowhere are these variables spelled out, and the study requires multiple groups (authors, librarians, publishers, research councils) to assemble the dataset.

The independent variable (the variable controlled by the researchers, in this case “open access”) also takes on multiple meanings. Treatment books are deposited into the OAPEN Library, although publishers are also encouraged to make them freely available from their own platform and any other platform they wish. Authors of treatment books are also provided with a digital copy and encouraged to put copies in their institutional repository and on their personal webpage.  These multiple treatments  makes it difficult to understand the impact of each one on such outcomes as print and e-book sales.  In this design, causes and effects become inseparable.

What is also unusual about this study is that it focuses on previously published books.  Monographs must have been published between 2006 and February 2011 in order to be eligible, with the experiment scheduled to commence in May 2011. This design rules out a study that begins with a cohort of newly published books that tracks their performance over their lifespan.

As Ronald Snijder explains in his open access study of AUP books, academic libraries (nearly exclusive purchaser of academic books in the humanities) do not make purchase decisions based on whether a monograph is available digitally.  Most university libraries rely upon approval plans that purchase books automatically from university and academic presses.  Hence, the decision on whether a book is available in PDF from a digital repository like OAPEN Library would make no difference to print sales of that book.  With regard to e-book sales, many formats are proprietary and function solely on a particular reading device.  As a result, a free PDF of a book is also likely to have no effect on e-book sales (assuming that a reader even finds that free PDF).

The experiment is therefore designed to reveal null results on sales, which is exactly what Snijder reports in his study, and as a logical conclusion, publishers should not be threatened by open access.  Coupled with increased PDF downloads as a result of putting free copies of books online — whether they are read or not — the headline of the study could be written today and the British government could have saved taxpayers £250,000.

The allocation of these research funds is also a little troubling.  Publishers will be compensated up to £6,000 (almost US $10,000) for each pair of participating titles, although the rationale for financial compensation is not given.  Publishers also select the titles, “participate in the collection and evaluation of the data,” and, as a benefit, “join the Steering Group to provide expert advice and guidance.”

Am I the only one squirming?

Since the methodology is not yet fleshed out in this experiment, this puts sponsors in a position of power to direct the analysis, interpretation, and reporting of the results.  This is like asking the pharmaceutical industry to join a drug study, allow them pick their own subjects, conduct their own statistical analysts, hire their own writers, and then reward them financially with public monies for participating.  The relationship between publishers and researchers has become uncomfortably close in recent years — especially in the UK —  and we should acknowledge that this may unduly influence the validity of results that come from such close bedfellows.

The debate over the merits of open access have moved into a new phase where case studies, economic models, and advocacy are not enough to influence public policy.  Yet if we turn to science to help provide us with answers, we must follow rigorous design and insulate these studies from the potential influence of special interest.  Given its methodology, the results of the OAPEN-UK study may have already been written even before the experiment has commenced.

Phil Davis

Phil Davis

Phil Davis is a publishing consultant specializing in the statistical analysis of citation, readership, publication and survey data. He has a Ph.D. in science communication from Cornell University (2010), extensive experience as a science librarian (1995-2006) and was trained as a life scientist. https://phil-davis.com/

Discussion

5 Thoughts on "Open Online Book Experiment Yields Expected Results"

I published scholarly books in the life sciences. Many years ago I launched a series. In so doing, I priced all books regardless of number or pages and illustrations and priced the launch books at $69.95. Each title sold withing about 20 copies of the other. The next year I priced the books which were in the same series thus same topical area at $105.95 sold the same. Within four years I was up to $169.95 and sold the same number of copies.

From the above I learned that the market for scholarly monographs is inelastic and that price really made little difference.

It is nice to learn that free versus not free is the same. The market is inelastic and very small. That those who want the information will pay for it. It seems to me that the same holds true for journals.

Additionally, I do not believe that OA will increase readership, but did open the door for many new publishers. That OA is very profitable and if it is not many of these new publishers will go out of business and with it their archive unless someone is willing to maintain it. Lastly, OA like many new markets, and OA is a new market, it will attract many hustlers and hucksters and many naive scholars will be taken in.

Dear Phil,

I’d like to respond to your blog.

• Link to report.
The first sentence of the linked page states clearly that we are talking about a pilot project in the Netherlands “Report about the OAPEN-NL project which explored open access monographs. The aim was to gain knowledge and experience of both the publication and funding of open access books in the Dutch context.”

The rest of the blog discusses OAPEN-UK. This means that the rest of the blog discusses another experiment. This causes the following discrepancies:

• “Matching subjects is a common experimental design in the social sciences that allows the researcher to control methodologically for variation that is known to exist between individuals, or in this case, books. Instead of testing for differences between each group of books (e.g. A, B, C vs. D, E, and F), the matching approach analyzes the differences between each pair of books (e.g. A-D, B-E, C-F).”
In the OAPEN-NL report, groups are tested, not pairs. The setup of the experiment is discussed in detail on in 6.4.2 The data set, starting at page 54.
• “What is also unusual about this study is that it focuses on previously published books. ”
In the OAPEN-NL pilot, newly published books are matched books that have been recently published. See the section 6.4.2 The data set, page 54.
• “assuming that a reader even finds that free PDF”
The report states on page 57: “When we look at the period August 2011 until July 2013, the 50 books of the pilot were downloaded 139,757 times.” So, at least some users did find the free PDFs.
• “The allocation of these research funds is also a little troubling. Publishers will be compensated up to £6,000 (almost US $10,000) for each pair of participating titles, although the rationale for financial compensation is not given.”
Firstly, the maximum funding per titles is €5,000, see 8.4 Funding on page 63. Furthermore, a large portion of the report discusses the costs of monographs. See 6.3 The costs of monographs in the Netherlands, starting on page 39. More detailed information can be found in Appendix 5: Costs per book and Appendix 6: Costs per page.
• “Since the methodology is not yet fleshed out in this experiment, this puts sponsors in a position of power to direct the analysis, interpretation, and reporting of the results.”
The OAPEN-NL report contains hopefully more information and data.

We’d be happy to discuss our report once you’ve had a chance to read it.

Ronald, thank you for responding. I may have made several errors due to my misunderstanding of your report and its relationship to the OAPEN monograph study. My apologies. Let’s try to clarify the differences between OAPEN, OAPEN-NL and OAPEN-UK:

1. Is this latest report an expansion of the paper you published in Oct, 2010 in Learned Publishing (“The profits of free books: an experiment to measure the impact of open access publishing, v23 n4 http://dx.doi.org/10.1087/20100403)? If so, what new information does it report?

2. What is the relationship between OAPEN, OAPEN-UK and OAPEN-NL? Are the studies using the same methodology? Are they working with different collections of monographs?

3. Do you know when the other reports are scheduled to be released?

–Phil

Dear Phil,

I’m glad that we can rectify any misunderstandings.

Regarding you questions:
1. No, it is not an expansion of that paper. However, it is set up in a similar way: an experimental group and a control group. The books in both groups are as similar as possible. The online usage is measured during the same platform (Google Books) and sales data now comes from several publishers, instead of just one. So, a different set of books, the experiment is conducted in another period, but the setup is the same.

However, the Dutch pilot contains more: research on user needs and project evaluation, and the costs of monographs in the Netherlands.

2. From 2008 until 2011, Amsterdam UP coordinated a project co-financed by the European Commission called OAPEN – an acronym for Open Access Publishing in European Networks. In 2011 the OAPEN foundation was created to carry on with the activities of the OAPEN project. The OAPEN Foundation – based at the National Library in The Hague – develops Open Access models for books and cooperates with academic publishers and research institutes to build a collection of Open Access books through the OAPEN Library. OAPEN is currently involved in two pilot projects in the Netherlands and the UK experimenting with Open Access monograph publishing. The results of the Dutch pilot project have just been released.

The Dutch pilot and the UK pilot are run by different teams and use different books.

3. OAPEN-UK is due to run until spring 2015. Apart from looking at sales and usage, it contains a research programme consulting with all stakeholders to gather data on attitudes, perceptions, challenges, opportunities and practical workflow related issues that relate to a move to open access monograph publishing. See http://oapen-uk.jiscebooks.org/research-findings/ for preliminary results.

-ronald-

The results must surely be influenced by whether any of the books are available as part of ebook aggregations such as those sold by UPCC (Project Muse) and JSTOR via subscription. Books published “open access,” if also part of these packages, will of course not suffer loss of sales.

The experiment we ran at Penn State Press in a Romance studies monograph series, providing free access to the complete texts online while selling print editions through POD, during 2000 to 2009 when I was director did not have such positive results. Compared with sales of print editions of the same series when it existed only in print from 1990 to 1999 were better, on average, than sales of the similar books in the series when it was available OA. Did sales to libraries in general decline between the two periods? Perhaps to a certain extent, but as I recall statistics from the ARL, after severe declines in the 1970s and 1980s, academic library purchases of monographs began leveling off in the 1990s and have been pretty stable since.

Comments are closed.