In a few weeks, Clarivate Analytics will release their 2016 Journal Citation Report (JCR), which will disclose the Impact Factors of over ten-thousand academic journals.
With each release, the JCR also suspends titles for citation practices that distort their Impact Factor score and rank. Last year, 18 titles were suspended from the JCR, 16 for high levels of self-citation, the other two for “citation stacking,” a behavior that is more informally referred to as a citation cartel. In prior years, the JCR suspended many more titles. In 2012, a total of 65 titles were suspended. In 2011, it was 50 titles.
The Impact Factor is a lagging performance indicator — a measure of last year’s citation count to papers published in the preceding two years. If you’ve identified that citation distortion has already taken place in your journals, as a publisher, there is little you can do but wait for your day of reckoning and hope that your journal escapes suspension.
Last year, Retraction Watch covered the investigation of a soil scientist, Artemi Cerdà, who allegedly coerced authors to cite his own papers and journal, Land Degradation & Development (LDD), for which he served as Editor-in-Chief before being forced to resign.
The collective effect of his coercion resulted in a massive increase of LDD’s Impact Factor, from 3.089 in 2014 to 8.145 in 2015, according to the JCR. Some of this rise can be attributed to self-citation, which accounted for one-third (33%) of its score. (Before Cerdà assumed the role of Editor-in-Chief in 2013, self-citation accounted for just 1% of LDD’s Impact Factor.) The other contributing cause of LDD’s score was citation stacking.
Using JCR’s cited-by tables, I was able to plot the flow of citations among four journals with which Cerdà was involved. This type of graph is called — somewhat befittingly — an alluvial plot, named after the sediment patterns made by moving water.
Among these four journals, we find a high flow of citations made in 2015 to other papers published within the same journal (top panel). This pattern of self-citation is not uncommon among specialist journals, as related papers are often published within same journal. What IS surprising, however, is when this flow of citations is focused on papers published in the prior two years (bottom panel) — the window from which the Impact Factor is calculated.
Self-citation and stacking were responsible for doubling Land Degradation & Development‘s 2015 Impact Factor
In 2015, nearly half (46%) of self-citations in LDD were focused on the prior two years of publication, compared to just 4% of LDD citations made to other journals. You can see this flow of green citations in the bottom panel of the above figure.
Even more extreme was the flow of Impact Factor-directed citations from Solid Earth to LDD. In 2015, the journal sent 350 citations to LDD, 68% of which were focused on the prior two years. In sharp contrast, Solid Earth only sent a total of 58 citations to itself.
So, why didn’t the Journal Citation Report suppress LDD and Solid Earth in 2015? To consider a journal for citation stacking, the JCR measures:
- Donor journal citations as a percent of recipient’s total journal citations.
- Donor journal citations as a percent of citations counted toward recipient’s Impact Factor.
- The number of donor journal citations counted toward recipient’s Impact Factor.
- Proportion of citation exchange between the donor and recipient journal.
In an email response from Stephen Hubbard, Content Team Lead for the JCR at Clarivate Analytics, Solid Earth and LDD met the first three criteria but failed the fourth.
While I appreciate that suppression decisions are based on a multiple criteria, the combined effect of self-citation and stacking was significant. Without them, LDD’s 2015 Impact Factor would have been just half (3.982) of the score it received (8.145).
Based on raw calculations, if LDD is included in the 2016 JCR, it will receive another Impact Factor score above 8, and self-citation will account for almost half (49%) of its score. I also count a total of 207 Impact Factor-directed citations from Solid Earth to LDD, which will account for 19% of its score. We should also remember that the investigation by the European Geosciences Union reported citation coercion in other journals as well.
There should be no question of the cause and purpose of the manipulation. We have have a smoking gun. We have resignations. It never gets clearer than this.
I work with editorial and publication boards who routinely ask me about strategies to increase their Impact Factor scores. While questions of self-citation and stacking often come out obliquely, some editors get incensed when they believe that their competitors are getting away with it, often flagrantly. Citation manipulation is therefore justified as a way of leveling the playing field. Personally, I find this line of thinking pernicious as it normalizes unethical behavior. Even coercion can be acceptable just as long as you don’t commit too much of it.
To me, the inclusion of LDD and Solid Earth in the 2015 JCR sends a strong message that citation manipulation is acceptable behavior just as long as it doesn’t reach a critical level — a level that is set absurdly high. Second, in the case of LDD and Solid Earth, there should be no question of the cause and purpose of the manipulation: We have have a smoking gun. We have resignations. It never gets clearer than this.
This leaves us in a quandary on what should happen next. Should LDD and Solid Earth be suppressed in the forthcoming 2016 JCR? If so, is it fair that the journal continue to bear the responsibility of the past editor? If not, has the damage to the reputation of the editor and offending journals sufficient punishment? And, most importantly, has Clarivate’s criteria for suppression changed the way we think about citation manipulation?