Years ago I was at a meeting that featured a talk by Cliff Lynch. I don’t remember where it was, and I don’t remember most of what he said. But one of his comments, which seemed (but may not have been) off the cuff, really stuck with me: “The purpose of collaboration is not to collaborate.” This principle had never occurred to me before, but once expressed in that way it seemed perfectly obvious: collaboration is something we engage in not for the purpose of “being collaborative,” but for the purpose of accomplishing tasks and solving problems that are best accomplished and solved in a collaborative way. And it then occurred to me that there were other concepts and principles that, similarly, are really means to an end but are sometimes evoked as if they were ends in themselves: innovation is one; change is another.

And another is disruption, which brings me to the topic of today’s post. But first, a little bit of background:

As many Scholarly Kitchen readers will know, Clarivate recently suspended indexing of eLife in its widely used Web of Science database. Clarivate explains its reason for doing so in an addendum to eLife’s entry in the Web of Science Master Journal List. Search for “eLife” there, and it comes up with the tag “On Hold”; click on that tag and you get the following explanation:

Concerns have been raised about the quality of the content published in this journal. The journal is being re-evaluated according to our selection criteria; new content will not be indexed during the course of the re-evaluation.

When the evaluation is complete, the publisher will be informed of the outcome and the journal will either:

• be removed from coverage if it no longer meets the quality criteria 
• or remain covered if it continues to meet the quality criteria. 

If the journal meets the quality criteria, any missing content will be indexed. If the journal is removed from coverage, content will not be backfilled. In the most serious cases of breaches in a journal’s editorial standards, as determined by our selection criteria, published content may be removed from Web of Science. Timeframes for completing a re-evaluation will depend on the particular circumstances of each case.

More specifically, the quality-control issue that led to eLife’s suspension was the fact that in 2023, eLife changed its publishing model such that it no longer makes publication decisions based on the outcome of peer review. As explained on its “Editorial Process” page, eLife subjects submissions to an initial review by editors; a paper deemed worthy of review is then sent to peer reviewers, and is subsequently published alongside its reviews, regardless of the reviewers’ assessment of its quality or validity. This practice falls afoul of Web of Science’s longstanding policy of giving “cover-to-cover coverage” only to journals that reject articles that don’t pass peer review. Clarivate uses the term “validated” to describe articles that have successfully passed peer review; journals that do publish non-validated articles alongside validated ones can get partial indexing as long as they are able to “provide a feed of content that is limited to articles that have been validated by peer review” – but these journals are not eligible for a Journal Impact Factor (JIF).

The response from eLife was largely defiant: “This decision from Web of Science stifles attempts to show how publishing and peer review can be improved using open science principles,” the platform provider said in a public statement the day after the suspension went into effect.

Subsequently, Clarivate notified eLife that its evaluation had led to the conclusion that “the cohort of content considered to have been ‘validated by peer review’ passes our 24 quality criteria. Therefore, we can continue to cover eLife in the Web of Science Core Collection. As per our policy, eLife will be partially indexed in ESCI, subject to the provision of the appropriate feed” (i.e., a feed limited to content that has successfully passed peer review).

Now, I’m not interested in litigating the upsides and downsides of traditional peer review here. This is an issue that deserves to be discussed, analyzed, and evaluated from multiple perspectives, and there’s no reason to expect (or hope for) universal agreement as to exactly how peer review should be done, or who should do it and when, or even whether it offers a net benefit to the scholarly community and the wider world.

What I do want to discuss, briefly, is a very interesting reaction from the Declaration on Research Assessment (DORA) that was published this week under the title “Clarivate’s actions regarding eLife: DORA’s response.”

In this document, DORA invokes the need for “constant innovation and renewal” in scholarly publishing, characterizing innovation as being “at the core of eLife’s policies and processes,” and accuses Clarivate of seeking to “(punish) innovation,” “shut down innovation,” and “hold the academic community to ransom.” It praises eLife for having “disrupted the traditional model of scholarly publishing” and laments Clarivate’s move as exemplifying “the overt challenges to disrupt and innovate in the scholarly publishing system.”

At no point does DORA’s statement address the substantive issue at the heart of the dispute between eLife and Clarivate.

In other words, DORA’s statement takes it as given that “innovation” and “disruption” are worthy ends in themselves, and that Clarivate’s position is invalid because it represents a lack of support for those things. At no point does DORA’s statement address the substantive issue at the heart of the dispute between eLife and Clarivate, which is about quality and validity in scientific publishing, and the effectiveness of eLife’s publishing model in helping to safeguard it. Instead, the statement focuses entirely on eLife’s value as a “disruptor” of and “innovator” in the existing system – as if disruption and innovation were somehow, in themselves, ultimate goods that outweigh any possible negative outcomes particular attempts at innovation and disruption might lead to.

Similarly, I was also struck by the way that DORA’s statement framed Clarivate’s actions as simply a reactionary response to a threat of disruption, rather than as a critical response to a genuine issue that needed to be addressed.

But this is a common problem, one that I’ve encountered among library managers and leaders from time to time throughout my career. Think about all the times a person in your organization has resisted change and immediately been dismissed as someone who just “hates change.” Of course, that may be accurate; the person in question may be genuinely reactionary. And it may also be that the person in question is perfectly in favor of change in principle, but objects to this particular change based on actual critical analysis of the reality on the ground.

Because obviously, there’s no such thing as “change” in the abstract. There are only specific changes to specific things, which may be for the better or for the worse. To defend the latter on the basis that they constitute change and that change is good by definition would, obviously, be ridiculous. And yet it’s all too tempting to do just that when we have conferred halos on particular words and concepts – “change,” “innovation,” “disruption,” etc. – and then to respond in a reflexive, knee-jerk manner whenever we encounter resistance to a particular example of change, innovation, or disruption. The problem, of course, is that when we react to resistance in this way, we lose the opportunity to hear what may be important insights about the implications of our proposed plan of action.

Circling back to DORA, eLife, and Clarivate: if you really believe (as both eLife and DORA claim to) that the Journal Impact Factor is “increasingly irrelevant” and is not needed to “show the impact that [journals] have within their communities,” then fair enough – you’ll find many who agree with you. Publish in whatever way you believe makes the most sense, and don’t worry about whether you have a JIF. But this is one of those things that you can’t really have both ways: if you care about having a JIF, then you have to publish in a way that doesn’t undermine the internal validity of the JIF. If you don’t believe that’s the right way to publish, and publicly proclaim that you “have never supported the Impact Factor and never wanted one,” then getting outraged about being denied a JIF doesn’t seem to make much sense.

Personally, I wish godspeed to eLife in its publishing project and continue to be happy to see a diversity of publishing models flourishing in the scholarly communication ecosystem. And I don’t see how eLife and DORA can possibly object to Clarivate’s decision not to fully index eLife and give it a JIF, given that eLife’s publishing model falls well outside the longstanding, well-publicized, and clearly explained criteria that Clarivate has established for indexed publications.