I am just back from the annual PSP conference in Washington, where I had the opportunity to attend a number of stimulating sessions. You can find the full program here. Of particular interest to me was “Plenary #1: A Tale of Two Continents — Open Access in Europe and the US.” There were three outstanding presentations — by Rachel Burley of BMC/SpringerNature, Amanda Click of American University, and Richard Wilder, Associate General Counsel at the Bill and Melinda Gates Foundation — but toward the end I felt a question creeping into consciousness; and that question was,
How do you evaluate the effectiveness of your programs; and when those evaluations are completed, will you make them openly available to the public?
The panelists did not have an answer to this question, and they freely admitted so: their candor was admirable and engaging. But still, I would have liked to know not just what they are doing but why.
Before digging in any further, let’s be clear that this question is not about the panelists. They did a good job. They are obviously people of high accomplishment who communicated with clarity and precision. The panel is not the subject here; rather, the panel is the occasion of my own reflections. Let’s not indict them when all that they are guilty of is intelligence, education, discipline, and attainment. We should leave the ad hominem arguments in Washington where they belong.
It seems to me that Burley had a good response to my question at hand, but she did not reach for it. As the only representative of a for-profit organization, she could have said that the evaluation of her program lay in the financial results: profitability, return on capital, trailing and forecast growth. I think she would have gotten very good grades by these measures, but she noted instead that BMC tracks such things as the number of downloads and evidence of engagement among users. But to what end? What does the number of downloads tell us, and what is the meaning of a tweet? (Please send your answers to @jospehjesposito.) Wherefore open access?
Click is responsible for an APC program at American University. It is a trial program, which is having the intriguing problem of not being able to use all the money allotted to it, as not all Gold OA venues subscribe to the protocols that Click’s fund requires. This is a tactical program; the rationale for the program lies elsewhere in the university. And so I continue to ask, wherefore open access?
With Wilder we move fully upstream to a major source of grant funding. In response to my question, he said that he did not have an answer, but that his area, like all units of the Gates Foundation, attempts to align its activities with the overarching mission of the Foundation. I wonder if I was the only person in the room to note that the Microsoft fortune that is behind the Gates Foundation derived from a philosophical position that is the diametric opposite of that of open access. It’s worth reading about the young Bill Gates’s “Open Letter to Hobbyists.” This does not mean that people can’t or should not change their minds, but it would be nice to know why. In any event, it seems odd that the foundation set up by the most results-oriented businessman of our age would not have developed a way to measure its activities.
If the panelists — and, more generally, the open access (OA) movement — were to offer a basis for evaluation, I think it would sound something like this:
Knowledge proceeds through communication, as one researcher builds upon (stands on the shoulders of) the work of others. Paywalls interfere with this sharing principle and thus slow down the pace of scientific discovery. OA will increase scientific communication and thereby accelerate scientific discovery.
Unfortunately, there is no way to do a controlled experiment to test this. What would the world of cancer therapies look like in ten years if everything were made OA today, and what would it look like if nothing were OA? Can we agree on a proxy for this experiment? Or are we simply going to take for granted an outcome that cannot be tested and for which there appears to be little curiosity about what the appropriate measures would look like?
There are some interesting corollaries to the proposition that OA will accelerate discovery. For example, some fields are moving to OA faster than others, which raises the prospect that in ten years we may have made enormous progress in (say) artificial intelligence, but not in cognitive psychology. I would like to know what the world will look like if the horses all leap from their gates at a different time.
Which brings me to the question that has nagged me since I first encountered Stevan Harnad’s “subversive proposal” in the 1990s: Why does anybody think this is true? It is not surprising that our panelists could not say how their OA programs are to be evaluated. OA has sat outside the realm of accountability since its inception.
I have remarked before that OA is a bad idea whose time has come. As a DNA-level pragmatist, I focus more on its inevitability than on its ostensible virtues or limitations. But I continue to ponder why anyone thinks it will work — what the evidence is for that — and why an entire ecosystem based on reputation and demand is being overturned in the absence of a method of evaluation. I am still waiting for that evaluation to be made freely and openly available.
Discussion
56 Thoughts on "Evaluating Open Access Programs"
The question seems to me to be which research has contributed to significant advancements. Who has tracked major breakthroughs or advancements back thru the literature. Since OA in any form is relatively new, that answer for these publications may not yet be evident.
On the other hand, there is evidence that the number of publications and impact factors are increasingly “signals” for those that evaluate individuals and programs for funding and for promotion and tenure “accepting” that peer review” provides some standard of quality of the articles but not the significance whether OA or not. The question here is whether OA contributes to the dilution of the value of articles as the pressure to publish accelerates with the decrease in tenure track positions and funding. This pressure is increasingly being seen in Africa, for example, as the World Bank and other evaluative bodies push universities to seek “world class” status as measured by published articles, in part.
Joe, One of the things that I like best about the APE conference in Berlin each January is getting a view of how different Europe is from the UK is from the US, in terms of science policy and publishing practice. This past January’s agenda had quite a bit on evaluation of OA on the agenda, but it was from the public-policy perspective (i.e., not from the perspective of publishers, editors, authors or readers in particular). My sense in retrospect is that metrics were output metrics (e.g., % of things that were open) not yet result (outcome?) metrics.
The agenda is here:
https://www.ape2018.eu/full-program/program
and the session videos are usually posted about two months afterwards (so not yet). These are the sessions related to your topic — some open access, some open science:
The UK, Europe and Scholarly Publishing: Funders, Authors, Employers and Publishers
David Sweeney, Director Research and Knowledge Exchange at HEFCE, Executive Chair Designate of ‘Research England’
The establishment of UK Research & Innovation, the body which brings together disciplinary Research Councils with our Innovation Agency and Research England as the block-grant funder in England, will change the environment for scholarly publication in the UK. Under development is a new Scholarly Publishing Licence (similar to the Harvard licence in the States) and the Research Councils Policy Review (due imminently) will now be carried out as a collaborative activity within UKRI. This will take account of progress towards open access since the Finch Group met in 2012 and also of progress with the Research Excellence Framework Open Access arrangements.
Keynote
Innovation with Participation – why we need Open Science
Prof. Dr. Johannes Vogel, Chairman, EU Open Science Policy Platform (OSPP) and Director General, Museum of Natural History, Berlin
Open science, participatory innovation and a scientifically literate citizenry are key pillars of a 21st century knowledge society and democracy. Science has to change in order to address the challenges arising from such assumption. The relationship between science, society and politics need to be recalibrated – the biggest changes are probably faces by the science establishment.
Current trends and movements arising from within science itself (from open publishing to citizen science), but also processes and imaginations at EU level and in the EU Commission hint already at forthcoming and substantial change. While probably being affected by the transition, today’s young scientists will have to be the drivers and implementers of change.
The EU Open Access Policies – from Vision to Action
Jean-Claude Burgelman, Head of Unit: Open Data Policy and Science Cloud, Directorate A – Policy Development and Coordination, DG RTD, European Commission, Brussels
My sense in retrospect is that metrics were output metrics (e.g., % of things that were open) not yet result (outcome?) metrics.
This is really key — OA is presumably a means to an end. But if you measure the effectiveness of the means by counting how often the means are used, it becomes self-reflexive, OA for the sake of OA rather than some greater goal.
Great essay. Evaluation and Accountability will come when the accountants say: You have no more money to do this! The grants are used up and the granters are no longer willing to give grants. When authors wake up and say why am I paying to have this article published when I can have it published at the expense of others than myself. When the government comes to its senses and no longer requires researchers to take part of their research grants to publish something that few will read and applies its tax income to roads and bridges. Finally when the audience says most of this stuff I am reading is trash!
I don’t think so, Harvey. I don’t agree with your assessment of OA, and it is pretty clear that you are sailing into the wind.
Ever been in Africa? I have seen students and researchers in Senegal, Ethiopia, Morocco and South Africa (yes, even South Africa) trying to get hold of recent scientific publications. For many people in developing countries Open Access is the only way to read a tiny fragment of what’s recently published. So yes, for a large part of the world “paywalls interfere with this sharing principle and thus slow down the pace of scientific discovery. OA will increase scientific communication and thereby accelerate scientific discovery.”
Right. The Gates Foundation’s home page states that they’re working to reduce inequity, so yes, access to research for those who can’t afford the paywall reduces inequity. It helps build capacity in low and middle income countries.
But without a method that states xx lives were saved or yy innovation was discovered zz years faster as a direct result of access to a particular set of papers, it appears some will question who the value of free access to research.
Interesting article. As you know publishers come in all shapes and sizes but one thing is certain the accounting and financial officers all know what OA renenue and expenses are in their organization. While publishers may not want to share that data, rest assured that they are on top of that data. Springer Nature’s accounting controls are one of the best in our industry. They have full financial data on the costs of each title and their OA program. Often that data is not always shared with sales, marketing and program staff even within their own organization but it is there and carefully monitored.
Let’s not despair that a controlled experiment is not possible as we have plenty of assessment and evaluation methods available that are used when said experiments are not possible.
But, what’s key in any process though is a clear articulation of the goal one is seeking and, as Roger pointed out last year (https://scholarlykitchen.sspnet.org/2017/05/23/open-access-scholarly-communication-defining-success/), there isn’t a single goal at play. I daresay that it is likely that different players are pursuing OA activities for reasons that are even conflicting (e.g., we have OA initiatives that aim to preserve commercial publishers and those that aim to put them out of business). What I think we must also avoid is assuming that the current system is achieving any particular goals either. The status quo/historic practices should get no more leave from being questioned as newer initiatives.
I do agree that there is a need for greater measurement and assessment. I’ve spent countless hours helping librarians, publishers, platforms, etc. work towards models of articulating goals and aligning measures in the past decade but there is still much progress to be made in clarifying purposes. I find measures are relatively easier to identify than intentions. All too often the reasons seem less strategic and more going-along-with-the-trends (which isn’t necessarily bad but may not be good).
Rick Anderson wrote on this subject as well:
https://scholarlykitchen.sspnet.org/2017/01/24/diversity-open-access-movement-part-2-differing-goals/
I think our discussion of the danger of face validity (from about a year ago) is relevant here, too. It makes a lot of sense on its face that if all barriers to access and reuse of scholarship were removed, scholarship would advance more quickly and robustly. So the temptation is to make policy based that obviously sensible proposition. The problem is that, like all obviously sensible propositions, this one may or may not be true, depending (among other things) on what you lose when you take away access tolls, and on what other knock-on effects there may be from messing with an established system. None of this is to say that the hypothesis is false; it’s only to say that face validity isn’t the same thing as actual validity.
One possible response to this observation, of course, is to say “Well, since the proposition makes obvious sense and since rigorously testing it would be nearly impossible, let’s just move forward as quickly as possible on the assumption that it’s true.” But to me, that seems like saying “Well, I don’t have a flashlight and I don’t have a good way of getting one, so I might as well just run into this dark cave as fast as I can”–whereas it would probably make more sense to enter the dark cave even more carefully than you would if you did have a flashlight.
Wouldn’t one measure of the success of OA be the proportion of people who want to access an article vs the proportion that can actually access it? For example, if everyone who wants to read an article can get it through via a subscription, then OA adds very little. We can see that’s not the case, as attested by the existence of #icanhazpdf, unpaywall (http://unpaywall.org/), and SciHub, but what’s the proportion?
No, that is not the measure. The measure is not who reads what but what new developments take place, and how do we evaluate those developments. BTW, Unpaywall is a legitimate service, so it is misleading to put it into the same argument as Sci-Hub.
Perhaps nobody evaluates open access programs because the value of OA is taken as self evident (that anyone can access the literature)? If that’s the value, then the measure of success is who now accesses it when they couldn’t before. If that’s not what you’re driving at, I may be misunderstanding what you mean by ‘developments’.
PS I’m aware that unpaywall is legit, I put those three together as they’re all ways to get hold of articles when they’re easy to get to.
Joe’s response is on target. We can start with all articles, OA or standard and look at the number of articles that are NOT read or downloaded and which, in their history have never moved the meter one notch further. Then, as I have suggested, OA vs Standard paywall articles can be tracked to determine whether OA has a better record of having someone stand on the shoulders to advance knowledge. OA needs a history which is not extant at the moment. But perhaps when the data is in then the value of the drive to publish/perish to signal funders and hiring/promotion evaluators to seek other metrics.
Again, that gets into a self-reflexive argument — the value of OA is that more stuff is OA, rather than that having more stuff OA does something positive for fields of research.
Probably worth keeping in mind too that “use in future research” is only one kind of use. It’s quite possible that the value of open is more in influence on policy-making or practice. Difficult to track areas, which altmetrics attempt to get at (though do so – in my opinion – rather poorly because the easier to track data points are not great representatives for those uses).
The difference between want/have subscription access vs want/have open access would be – in my view – not a horrible measure. However, the issue I see is that most of the time anyone can have access without open access – it just takes money. So, what you are measuring is actually wealth and willingness to spend … and maybe access to formal/legal sharing networks like interlibrary loan. At which point we might ponder if our goal is something different than anyone who wants/has access but somehow remedying wealth differences, which is quite a different purpose (and might point us to different metrics)?
Lisa raises two critical points:
a) wealth as a condition for access is a major issue as many of us who work in developing countries know well whether on the subscription or use side and again via the cost to publish on the OA side. This, of course, is one of the reasons that pirate hubs exist or vehicles for alt publications. This issue is one that Robert Maxwell understood well when he built his academic empire based on the idea that academics, themselves, via review processes, act as effective ways to create “value” and thus profitability.
b) the policy issue is a bit of a red herring. Publications, particularly in the STM area, were originally designed for communications to exchange and advance scholarly research. That should be considered in the social sciences also. The issue is problematic, particularly in economics but also shows up in the biomed area where certain editorial and review vetting have philosophical bias leading to publication preferences. This, of course, is a long and complex discussion separate from efforts within the STM community attempting through academic journals to influence policy outside of the community. Global warming research may be the current paradigmatic example.
A couple of thoughts:
1. The most important consequence of OA may be its effect on information diffusion to non-scientists:
http://onlinelibrary.wiley.com/doi/10.1002/asi.23687/full
2. Experiments in this *space*, if not in the exact way you describe above, are possible:
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3039505
Again, is that self-reflexive — the goal of OA is to have more OA stuff for people to read. Shouldn’t there be an outcome we’re seeking by providing that material freely to non-scientists (and to be clear, OA goes way beyond just the sciences)?
Would a measurable goal be an increasing level of scientific (and other) knowledge among the general public? Or is that still too abstract?
Web of Science now identifies OA articles published in any journal. Here is a brief analysis of the Impact of Open Access Papers in Hybrid Journals https://pubs.acs.org/doi/pdfplus/10.1021/acsenergylett.8b00061
Despite the larger readership there is no clear cut advantage for OA papers to garner citations. Subscription journals continue to attract high impact papers.
My inclination, without any hard evidence to back it up, is to suggest that (1) the rate of discovery and advancement of learning is not in any major way enhanced by OA because most of the most important advances in science are achieved by researchers associated with universities or other organizations that can afford to provide toll-access journals to them but (2) the wider availability of scholarly literature worldwide does enhance the spread of education to more people, including people not associated with academe. Regarding #2, the analytics that show geographical distribution of articles of mine available OA persuade me that they would never have reached people in so many different places via just toll access and publication in print form. When you consider that most scholarly monographs published by American university presses are held in library collections in the US by just a couple of hundred libraries, it stands to reason that making them OA will spread access to them enormously. What effect, if any, that access has on the advancement of science and other fields is a separate question, but I believe there to be value in the enhancement of educational opportunities just by itself.
another excellent piece that points at the reliance of OA advocacy on religious truths that at best have yet to be demonstrated, at worst have evidence contrary to them ignored or disparaged.
a lot of the “metrics” people propose above are not very useful. is it important, for example, who can get access to “scientific” publications (although OA covers much more than science, which is part of what worries me so much), or that *people who are engaged in work that contributes to that science* can access them?
what is the benefit of people who don’t do science, or aren’t part of any given profession, in being able to access professional work product they don’t need for their work?
does that benefit outweigh the myriad ways in which having a professional publishing system benefits those who do work in the given area?
does that benefit outweigh the imbalanced economic situation in which power corporate players can access huge amounts of fundamental scientific research for free, while in no way returning the favor?
the destructive effects of OA–which turn out to be much more focused on the non-corporate players, such as scholarly societies and university presses, as opposed to corporate players like Elsevier that can pivot and use OA discourse to their advantage, and lawyers to clean up the messes–are significant. Unless we are willing to calmly and objectively address the pros and cons, it is hard to see how we can consider the OA effort entirely honest. and the fact that so many of the strongest OA advocates have strong institutional self-interest in promoting their “missionary” work makes them not altogether reliable reporters. OA’s benefits should be able to be stated in clear, unambiguous ways, with data to support them. As should its drawbacks. Resistance to these requests is prima facie evidence that OA advocacy is not, despite the advertising, about benefiting academic work.
Just to be clear: I don’t think there is any reason to impugn the motives of OA advocates. No ad hominem arguments, please.
I have to chuckle (and wince) when some commenters intimate that those committed to freely sharing research have somehow been duped by an OA Deep State — like geese falling for a laughable sham. Most in the research universe are skeptics and prefer evidence. When evidence and evaluation frameworks coalesce, they also rely on experience and judgment among other things to move forward in their pursuits. Open access has been around a while, so it’s not as if the Gates Foundation, Wellcome Trust, the European Union, Australian Research Council, Austrian Science Fund, National Research Council Canada, Research Councils UK, 600+ universities, etc. have been forced into making rash, uncritical decisions (or to run into a dark cave as fast as they can without a flashlight).
I’m not a scientist. In addition to regularly employing my critical faculties, I respect the judgment of those who dedicate their life to research. It’s hard to dismiss the international experts on the Gates Foundation’s Scientific Advisory Committee and the NIH when they urged the foundation to double-down on its commitment to accelerate open access to the research the foundation funds and the data on which that research is based. And to ignore their statement that the collective commitment to fostering open access to published research, open data-sharing, and ultimately open research collaboration could be the single most important innovation that [the] organizations [R&D funders] are remembered for a century from now (https://www.linkedin.com/pulse/its-time-revolution-data-access-trevor-mundel/).
A completely circular argument. BTW, why don’t you use your real name?
Hi Joe
I think the key, here, is the “Linkedin” article by Trevor Mundel from Gates posted by Lindy, along with the diagram showing the progress of a paper from posting along with relevant data to final article.
Basically:
a) It changes the nature of the initial screening, reducing the time before the article gets public exposure. This reduces the pressure of the editor and EAB and thus the restrictive gatekeeping of the journal. It does not, however, necessarily reduce or eliminate the predilection of the editor and EAB towards favoring potentially contrarian approaches of authors (examples abound in medicine and economics).
b) It favors post publication review which does place it in the public view and may open the area to critical scrutiny. As Lindy suggests, it could lead to collaborative exchanges
c) The elephant in the room is what does this mean for the value of the journal as a measure for funding of future research or a determinant in the “publish/perish” portion for promotion and tenure when the shift of credence now moves to the post publication review/revision path and becomes visible under OA. The journal editor and EAB directly or indirectly are now also subject to public comment with regards to what has or has not passed thru the initial gate towards post publication review.
Given just the current state of AI in reading documents, one might believe that the initial gatekeeping of the editor and EAB could become handled by Watson or off-spring while the EAB, should they choose, could add their critical analysis in public under post publication review.
Basically what the Gates Foundation with its associate participants are proposing changes not just the path to getting research out to a larger community but basically a return to the original idea when Phil Trans was first published in 1776.
Yeah, sorry about those messy sentences with a few missing words. I hope the substance was coherent enough.
I agree that the methods toward achieving open sharing of research need rigorous evaluation. It’s exciting to see all the experimentation occurring given that some methods aren’t sustainable (e.g. rethinking whether APC $ could be invested elsewhere to better achieve goals — recognizing that goals vary by researcher/funder).
I’m curious (in the purest sense — my goal is to learn) about when you mention that a system based on reputation and demand is being overturned. Can these elements still exist when research is freely accessible? Again, I’m asking a question — I really don’t know — rather than making any kind of argument. I haven’t read the majority of posts here, so if you’ve written about this, a link to a previous post would be appreciated. My curiosity isn’t necessarily about the value of reputation and demand, but rather their (lack of?) permutability.
The answer to your question is no.
Lindy,
If you go back to your early post where you cited Trevor Mundel, there is a diagram of how Gates is approaching OA. In the end, their materials are subject to open, post publication, peer review by parties selected by the editors. It is not clear if others can also post evaluations. This process is circular but not clear what signals that it is finally acceptable.
There are several other issues that have been touched on which may be applicable to both OA and conventional publishing and that Gates raises, indirectly:
a) Most, if not all of the articles in the Gates OA portfolio are from research funded by the Foundation or partners. This points to the idea that ports of submission do not need traditional selective criteria or have to adhere to the increasingly narrow focus, provided they are sufficiently “keyed” to be uncovered by increasingly intelligent search engines which also can uncover individuals who are qualified to review. There are search engine sites that, today, have this capability by even searching text. One does not argue that journals with selective criteria might offer an initial entry point. The ramification for publishers who increasingly enlarge their stable are worth considering
b) As mentioned in this thread and which is increasingly clear, is what is submitted for research results are increasingly of interest to many qualified individuals who require access to this knowledge but who are not directly engaged in advancing the research. They may or not have the fiscal resources or even access to the publications, as noted in this thread. As many in research understand, the work in which they are currently engaged may be significantly different from where they started. As noted in “a” where they get access and even where they choose to publish may cross traditional lines. Publishers and many academics have responded to this by creating more journals either narrower or more interdisciplinary. Again, changing abilities to access through intelligent search can go a long way in reducing such practices
c) Cited in this thread is a study of ACS journals comparing OA to conventional publishing. The Gates approach offers another element here. By having open, post publishing review exposes the article to a larger audience as well as the reviewer practices. This shines a significant light on the entire system, perhaps stronger than the traditional impact factor. It could go a long way to reducing intellectual persiflage before the material is accepted and not after it is accessed either through an OA or conventional publication. In other words, researchers may have to consider that their materials and the review thereof will be exposed, a potential way to quickly cull those articles as criteria for obtaining funding or for researcher advancement whether in academia or not. It will increase the burden on funders to understand the value of the research produced for further funding rather than the default or weak signal of the impact factor.
d) The ramification for publishers could prove interesting since fewer gates, journals, would be needed under the Gates system; currently less selective journals could be eliminated while opening up opportunities for more rapid access. The concerns about large publishers such as Elsevier adding/ shifting to services rather than selling “product” in the form of journals becomes of serious concern. It follows the shift in product platforms to service platforms seen outside of the academic community and could lead to the absorption or elimination of publishers in the scholarly research communities.
Joseph, your argument is a non-argument and ideologically charged. Why do you insist on measuring the success of OA in simplistic metric terms? Equitable access to scholarly output paid for by society is the key metric. I’m a librarian who is successfully involved in OA scholarly publishing at the deep end. We have thousands of full-text downloads every year. OA has its place in the scholarly ecosystem and in citizen science. And there is nothing you and your vested interests can do about it!
First, why don’t you use your name online? Who are you and why should I believe you are a librarian? Second, you did not read my post slowly enough. I made NO argument for metrics; I asked for a form of evaluation and want to know when the evaluation will be made public. Finally, I have no idea whatsoever what you mean by my “vested interests.” What could they be? Most of the work I do is with not-for-profit organizations, including libraries.
How can one effectively and “neutrally” evaluate, on a meta level, the worth and veracity of a largely taxpayer funded public good? Is it necessary or, indeed, constructive and desirable to question the indisputable value of OA that you keep pondering about? Questioning the veracity of OA only plays into the hands of commercial publishers who do not offer any “value” that existing academic library and scholar-led diamond OA publications already perfectly well deliver. And you do represent commercial interests (going by your bio) – hence my questioning of your motives here. An interesting excercise would instead be a philosophical podium discussion on this subject matter at a fitting conference. That is, a philosophical evaluation of OA (because you and I and everyone else are already paying regardless). What should be pondered over here more than anything else is the wider socio-cultural impact of OA. Meta-level evaluations other than philosophical and tangible case-study based ones are neither constructive nor conducive to the wider OA debate.
About three-fourths of my income derives from the not-for-profit sector, so toss out the issue of my vested interests, and while you are at it, let’s recall what we learned in high school about the inappropriateness of ad hominem arguments. In any event, you are simply wrong in saying that publishers add no value. If you really believe this, we have nothing to discuss. As for “the wider OA debate,” there is no debate. There is only one hymnal and no self-reflection.
Added value from publishers is, indeed, questionable. See my reply to the 102 Anderson points. https://scholarlykitchen.sspnet.org/2018/02/06/focusing-value-102-things-journal-publishers-2018-update/ . My evidence is that it is possible to establish and run a journal without them. All you need is a laptop, editing and minor technical skills, and an academic network. You certainly don’t need any money, but you need sufficient time. Publishers make things easier, but they are not essential.
Ah time.
Probably the most common thing I hear from researchers is that time is their most precious commodity. If you have all the time in the world, why hire a lab technician, just do all that scut work yourself. Why hire a plumber when the drains back up — just get a book and take the time to learn how to fix it yourself. With infinite time, why do we need librarians and the services they provide? Heck, why not fire all the groundskeepers at your university and tend to the lawns yourself? All it takes is time.
Also, to be clear, the time scales required for a journal that publishes around 30 articles per year is somewhat different than that required for a journal that publishes 200-300 per year, or one that deals with thousands of submissions annually. If one is truly successful, then at some point, volunteers don’t have enough time and one starts to need employees.
I mentioned scale in the other post. Many social science journals are not large, and many must surely be uneconomic for the commercial world, unless they charge a decent amount to authors. So it is understandable if some are set up by academic departments and so-on. Ours publishes 60 articles a year now and that’s about our limit. Several small OA journals do get a publisher as the numbers grow, or end up charging authors to pay for costs – the International Journal of the Commons is one example.
Walt Crawford’s work provides a quantitative analysis of types of publisher, APCs and so on for all the DOAJ journals in 2017. Esposito seems to miss it completely, and it is an evaluation of the field, with some clear criteria. He even gives his raw data. https://waltcrawford.name/goaj.html Raw data https://figshare.com/articles/GOAJ2_Gold_Open_Access_Journals_2011-2016/5023256
The essential info for me is on p13-13 of his latest (2017) report. A group of journals in DOAJ he labels ‘APCland’ comprises those of the publishers BioMed Central, BMJ Publishing Group, Dove Medical Press, Elsevier, Frontiers Media S.A., Hindawi Publishing Corporation, MDPI AG, Nature Publishing Group, Oxford University Press, Public Library of Science (PLoS), Springer and Wiley. Their average APC for 2016 was $1,867 . He says, “although APCLand accounts for one-fifth of the serious gold OA journals and less than two-fifths of the articles, it takes in more than four-fifths of the revenue.” The rest are OAland – “4,300 names in a list of unique DOAJ publisher fields…. These publishers accounted for 81% of the active journals and 61% of the articles, but only 17% of the revenues.” “.. not only do 80% of the journals active in 2016 not charge APCs or equivalent fees, those journals account for 65% of the articles .”” In other words, in OAWorld most articles—nearly two-thirds—did not involve author-side charges”. “for those articles that did involve fees, the average cost per article was $635”.
To qualify, maybe it is publishers in that APCland category that we should seek to avoid, if academics cannot do their own publishing.
We agree to disagree. And that’s fine. That’s what debate is all about! OA is a political football that it quite simply shouldn’t be. Your argument of abstract “evaluation” is quite simply unrealistic and nonsensical (and I’m pretty sure you know that very well). I have not come across a single commercial publisher delivering any extra value that a solid academic library (as publisher) cannot. The metrics game is flawed at every level – a fact that everyone is well aware of! And a fact discussed daily amongst librarians and academic colleagues. I speak from first-hand hands-on experience, as well as against the backdrop of commercial publishers trying to sell back great-deal “open access” packages. The debate around OA is a fascinating one, and some of the non-arguments are most puzzling.
I was not going to reply to your most recent comment, but the remark that “I’m pretty sure you know that very well” is an outrage. First you accuse me of having “vested interests,” now you accuse me of being a liar. If this is the best the academy can give us, god help us.
Can I ask you to address the question around your rationale of linking abstract “evaluation” of OA as a justification of its prevailing existence (or not, as you stipulate) in the first place (again, what’s your motive for linking the two?!).
I quote from your opinion piece above:
I have remarked before that OA is a bad idea whose time has come. As a DNA-level pragmatist, I focus more on its inevitability than on its ostensible virtues or limitations. But I continue to ponder why anyone thinks it will work — what the evidence is for that — and why an entire ecosystem based on reputation and demand is being overturned in the absence of a method of evaluation.
Your article’s core argument – the redundancy of OA – is still a non-argument unless you convincingly explain your rationale and motives.
I apologise if you are taking personal offense her. It’s not my intention to offend you. But you are not engaging meaningfully with any of my counter arguments (and I can start listing various studies to prove any of my points). Name one service, for example, a commercial publisher delivers that an academic library (or library consortia) cannot. You are not engaging on factual grounds, but merely on personal sensibilities.
Well let’s be fair, your initial comment did accuse the author (an independent consultant who I can assure you, largely works for not-for-profits) of representing commercial interests and you questioned his motives. It’s also kind of an odd approach in 2018 — the battle over OA is done, OA won. It is now firmly established as a major way that research is published, and has largely been happily embraced by those commercial interests. The biggest OA publishers are the big commercial publishers, and they are thrilled with the profits that OA is driving for them. It would seem counter-intuitive to still think there’s some anti-OA bias from the big corporations.
Name one service, for example, a commercial publisher delivers that an academic library (or library consortia) cannot.
How about a global end-user marketing workforce to get an author’s articles noticed by readers from amidst the enormous avalanche of research currently being published? Investing millions of dollars in technology development? Fund startups like CrossRef and all the Digital Science properties? I could go on but scale and investment are incredibly valuable tools for market leaders. It is unlikely that most universities, which vastly undervalue their libraries, are going to start giving them tens of millions of dollars to gamble with. Corporations take those risks all the time.
The problem with your argument (and this is coming from someone that works for a university press) is the assumption that what the current market leaders do is easily replicable by anyone, or that they aren’t doing anything that has appeal to their authors and readers. If you hope to displace them (as many of us do), you need to better understand just what it is that they’re doing that meets the needs of so many. Researchers are really smart. Librarians are really smart. They’re not getting tricked by commercial publishers. If you want to beat them, you have to out-compete them, and that may be a more difficult task than you appear to think.
Copyeditors? Last time I looked there were no trained copyeditors on library staffs. Marketing? What library has a marketing department? (if you think the need for marketing disappears in an OA world, you are sadly mistaken.) I could go on . . . .
P.S. As a former university press director who hired Joe as a consultant, I can vouch for his knowledge, analytic skills, and sensible advice.
Developmental editing might be even harder to accomplish than Copyediting. Oh — and what about advertising sales? Every penny you can make in ad sales is a penny less you have to charge authors, readers, funders or donors. Most libraries don’t have a lot of experience selling ads.
Thank you, Sandy.
Where academics set up and run a journal, without any publishers, we can obtain some evidence as to whether commercial publishing in that field is really needed. Here is my experience. I will not say the name of the journal since we don’t need the promotion, but you can look it up.
Copyediting; done by the editors. No budgetary constraints, since we are paid for other work (as scholars) or retired. No budget at all. Marketing – don’t do any since little is at stake. Commercial pressures: none. Staff none, software is now OJS, tech is a laptop. Site hosting in our case is a university, assisted by librarians, or we could pay privately at affordable rates.
On social justice criteria, I reckon not charging readers or authors is a pretty important gain.
We certainly don’t need commercial publishers for any of this.
Evaluation of success: various article citation measures (very favourable). A few complements received. More submissions than we can handle.
Since a rival journal has now been established by Sage, we will be able to track which works best over the next few years.
Some journal editors may have copyediting skills, but very few if any are trained as copyeditors and I’ve met a great many academics who don;t know a dangling modifier when they see one. If you’re satisfied with low-grade copyediting, then that’s not a problem.
I’m well aware how commercial presseses are subverting OA. What commercial presses can do libraries (i.e. consortia) can do. The question of scale is of course a massive challenge. Reverse engineering can go some way; we are talking about a public good after all and taxpayer money has no, or if at all, a most limited place in commercial publisher pockets. The argument of millions of dollars required to successfully run OA press in terms of marketing is not persuasisive. Librarians happen to know a thing or two about indexing, and we collaborate with and advise academic colleagues around publishing. And of course, I agree, marketing is important. Copy-editors? That’s one of a few real costs a commercial press spends money on just like a library press can. You assemble teams of expertise and hire people in. The funding model of library OA presses can be squarely (and fairly unlike the commercial presses) addressed (I’m happy to go into detail). I’m afraid to say (and I’m by far not the only one) that the problem is a systemic, practice-of-culture one. and not a practical. The power of the commercial press is immense. As said, I don’t mean any offense, but the article’s author does not, by reason of factual debate, make a convincing case. What is “evaluation” of OA on the back of what I have argued above? The author has not engaged with that question nor my critical input. What he does instead is question the value of OA on the basis of a non-argument.
First, let’s get this out of the way:
we are talking about a public good after all and taxpayer money has no, or if at all, a most limited place in commercial publisher pockets.
Governments frequently spend “taxpayer money” with private, for-profit companies. Who do you think builds all the roads or missiles? Further, much research in science is funded by private industry, and much is not funded at all. In the social sciences and humanities, much (most) research is not funded. In the US, library purchasing budgets are largely driven by tuition revenues and student fees. Limiting the ability to publish strictly to government entities would remove players like Harvard, MIT, Stanford, etc. from the landscape and likely result in something resembling Lysenkoism.
That said, I wish you all the best, and will be thrilled if you can succeed, but I think you’re grossly underestimating the undertaking. Being good at indexing does not mean you’re good at marketing or publishing. As someone who moved from the research bench to the publishing business, it is an entirely different undertaking that requires entirely different skill sets than those found in academia.
Probably the largest hurdle you’re facing is governance, which universities and libraries are not very good at (https://scholarlykitchen.sspnet.org/2011/10/24/governance-and-the-not-for-profit-publisher/). Compare, for example, how SHARE and CHORUS started simultaneously, and how quickly CHORUS was up and running and how long it took SHARE. Not-for-profits must run at a surplus if they hope to grow and survive. See for example the strong efforts in recent years from PLOS, running at a significant profit, which has allowed them to build new infrastructure and experiment with new ways of publishing (not to mention weather a downturn in revenue). Most libraries lack significant experience in running successful businesses. You should note that the two most successful university presses have done so by moving business operations out of the university itself and hiring business experts to run things, rather than relying on librarians. If this sort of outsourcing is indeed your plan, why not instead rely on the existing university presses?
As for this post, presumably there’s some overarching goal behind open access, other than open access itself. Will having more access to the research literature improve society or our lives in some way, or is it simply a self-reflexive activity, OA for the sake of OA? How will progress toward that goal, if it exists, be measured and evaluated. Those are not an unreasonable questions to ask (and are quite separate from questions of who should be running the show).
I’m questioning the author’s rationale of linking abstract “evaluation” of OA as a justification of its prevailing existence (or not) in the first place (again, what’s the motive for linking the two?!). I also wrote that certain areas of expertise can, and must be of course, brought in (like every business / library does). Your linking/comparison of road building and public contracts etc. with open knowledge sharing is simplistic and binary (no offense). The challenge is overcoming established cultures of scholarly communication practice and funding arrangements. May I suggest that the author takes time out (you’re invited too, David) to attend the 2018 Library Publishing Form in May (https://librarypublishing.org/library-publishing-forum/) to broaden perspectives and critical faculty, rather than blindly submitting to established norms and the present status quo!
The most sensible thing for libraries to do is not to try recreating all the functions of a publishing house within libraries but simply to join forces with the university presses that already exist on nearly 100 campuses in the US. This synergy is what has prompted a number of universities to do just that, which happened at Penn State in 2005 when the press and the library jointly established the Office of Digital Scholarly Publishing to do various types of OA publishing. Formal merger isn;t even necessary to achieve this synergy. The University of California Press achieved it simply by having the press and the California Digital Library cooperate on projects together. The Mellon-funded report titled University Publishing in the Digital Age (2007) had an appendix listing side by side the respective skills sets of libraries and presses to show just how such synergy could be realized.
If my reading of history is correct, OA was an idea that came out of silicon valley and was then implanted in some academics minds as a goal that would benefit society for a variety of reasons. OA, i believe has the greatest benefits for industry – who would have to pay nothing for access to all the latest research – which allows them to access the ideas and technologies from around the world with no outlay. Academics and governments have just been easily duped, by what seems to be a plausible argument for OA, that they are working for the greater good of science and huimanity.
I’m not sure where you got that idea from, but I can assure you that we were exploring approaches to OA publishing at Princeton University Press as soon as electronic publishing became feasible. We had a joint project in OA monograph publishing in the CIC (now the Big Ten Academic Alliance) under way in the early 1990s. I trace this history here: https://scholarsphere.psu.edu/concern/generic_works/x346dv41v
What is the problem that needs to be solved? It would seem that it is access to quality/qualified research results.
That problem and solution changes with time when comparing Phil Trans in 1665 and today. But more importantly “Today” in light of current technology is already changing. OA, as noted, is here. Going back to past materials in the “Kitchen” this is clear, particularly with the move from a product platform to a service platform and thus what publishers are selling. Gates’ approach makes this possibility visible as does the direction of Elsevier and others.
This makes the arguments of “for whom” and “why” moot. As also mentioned, publishers are now equally able to monetize both conventional and OA publishing, again raising questions of the function of the traditional scholarly journal in its current embodiment as well as the current business model and potentially the survival of some publishers- again a concern reflected in the Kitchen.
Let’s bring Simon’s argument forward. No one has addressed fully, is Robert Maxwell’s insight regarding the value of the scholarly journal purely as a business venture. One does realize that the act of publishing a quality journal takes time and/or cash in some balance. What lay behind OA was access by the fiscally disenfranchised such as in developing countries and even in the better funded research institutions with restrictive budgets being able to pierce the paywalls, legally. That has been addressed partially by shifting the costs to the researchers, presumably by charging the cost to the funding agencies. But that now has become problematic as the publishers have figured out how to compensate with, often, egregious APC’s which now reduces options for the same communities to publish. And it has raised concerns in other parts.
What Maxwell realized early on is that publishing in recognized and ranked journals was a signal as to the value of the article. And today, this has been established where funding agencies use such publications, as signals, like Indians in the western US in the old days counted “coup”, in order to fund; and, academics, never having evaluated these articles internally with their own faculty, use these same counts as a, critical, basis for promotion and tenure. Like Olympic “Gold”, the medallions now are gold plated. Thus, it’s not just the publishing but the ability of the publisher to promote in many ways the highest value whether it is a for or not-for profit publisher. Thus, the problem sought to be mitigated by OA has reappeared.
