If journals are like ships, then the Editor-in-Chief and the scrappy crew of associate editors are ultimately responsible for the leadership and direction of a journal. When the ship runs into trouble–an allegation of plagiarism or fraud in one of its papers, for example–editors are supposed to take control of the bridge, demonstrate integrity, fairness, and ultimately take responsibility for the reputation of the journal. If a journal were like a ship, the Editor-in-Chief and editorial board would not abandon their journal during a catastrophe. They would go down with their ship. This is their duty, and the world expects this behavior.
The equivalent of a shipwreck happens each year as Thomson Reuters suspends dozens of journals from the Journal Citation Report (JCR)–an annual publication that reports the Impact Factor for thousands of titles–for engaging in publication behaviors that distort the citation record. This year, 38 titles were suspended from receiving an Impact Factor: 23 for high levels of self-citation and 15 for “citation stacking” an ambiguous label to what most would consider unambiguously as a citation cartel.
In 2012, I reported on the first instance of a journal citation cartel: 4 biomedical journals with overlapping editorial boards that were publishing selective reviews of each others journal. These “reviews,” often labeled as editorials, delivered hundreds of citations to member journals in the cartel, significantly raising their Impact Factors–and their standing–among other titles in their field. At that time, Thomson Reuters, had no mechanism to detect citation cartels.
This year, most notably, six business journals were suspended from the JCR for citation stacking: Enterprise Information Systems (Taylor & Francis), Management Decision (Emerald Group Publishing), International Journal of Production Economics (Elsevier); International Entrepreneurship and Management Journal (Springer); Systems Research and Behavioral Science (Wiley); and The Service Industries Journal (Taylor & Francis).
The JCR Notices page provides some pretty shocking details on the flow of citations among these six journals. For example, 95% of all citations flowing from The Services Industry Journal to Management Decision in 2013 were focused on the prior two years of publication (2011 and 2012)–the window from which the Impact Factor is calculated. Similarly, 94% of all citations from International Entrepreneurship and Management Journal to Management Decision in 2013 were focused on the past two years. Taken together, these two journals were responsible for 71% of all citations counting towards Management Decision‘s 2013 Impact Factor.
While helpful in understanding why these titles were delisted from the JCR, Thomson Reuters does not reveal the publication strategy that led to their suppression. Did editors engage in publishing their own systematic reviews of each other’s papers? Or, did they coerce authors to do their bidding–a tactic that may be endemic to parts of the business literature. Whatever the cause(s), Thomson Reuters’ response was to delist all six titles for tactics that ultimately distort their evaluation and ranking of journals.
Delisting a journal from the JCR does not necessarily sink a journal, but it clearly does harm to the the profile of the title when reinstated. Cell Transplantation received a 2010 Impact Factor of 6.204 before being suspended in 2011 for citation stacking. It was reinstated in 2012 with an IF score of 4.422, which dropped to 3.578 in 2013. Medical Science Monitor, another title suspended in 2011, dropped from 1.699 in 2010 to 1.358 in 2012, to 1.216 in 2013.
And this is where the shipwreck metaphor fails. Editors rarely go down with their ship. Many simply continue their role after their journal is delisted. Others are quietly removed from the mastheads and asked to join other editorial boards if they are still perceived to hold any clout, or simply retreat to their offices to resume their academic careers of teaching and publishing. The real victims of journal suspension are not the captains, but the passengers–the authors who decided to sail with the journal.
It would be easy to blame the editors, for their willingness and participation in a citation cartel. But before we hold them accountable, we should at least acknowledge that their intentions–to draw attention to their journals–was not entirely bad. Editors are supposed to promote their journals and the articles published therein. How they accomplished this goal, however, is open to debate.
It would also be easy to blame the publishers of these six journals for providing lax oversight of their editors. But before we do so, we should be reminded that editors and authors quickly become incensed when publishers begin micromanaging their content, as in the recent controversy over an article published in the Taylor & Francis journal, Prometheus. Academics can be quick to blame organizations when fault is staring at them in the mirror.
Lastly, it would be tempting to just throw up our hands and blame the system. If you are an author who is evaluated by the journals in which you publish, discovering that you were just published in a delisted journal can be a horrible shock. Why should you be punished for the ineptitude of the captain and his crew?
Delisting a journal from the JCR sends a very strong signal to the community of editors of what Thomson Reuters is willing to do if they perceive that you are gaming their system. For most editors, the benefits are just not worth the risk, especially when your authors suffer so much collateral damage. Other editors, clearly, are willing to take that risk.
Is there a way to target those who are responsible for gaming the system without punishing innocent authors? I’d like to propose a different solution to delisting journals and would like to engage our readers in working through the benefits, drawbacks and unintended consequences of this change. Here it is:
Instead of delisting a journal (for extreme cases of self-citation or citation stacking), Thomson Reuters should flag the journal in its report, highlight the offending article(s), and assign an Impact Factor without the offending article(s).
Would such a solution be preferable than the current model? Or, by fixing the collateral damage problem, does it create a whole new set of unanticipated problems?