In times of controversy, it may be tempting for an author to remain silent and let others speak. Yet in science, silence is often an indication that something has gone awry which cannot be explained. Rather than placating the public, an author’s refusal to speak has just the opposite effect: Silence generates skepticism and undermines trust, not only of the paper itself, but of the authors, their institutions, and the journal that published their work.
After weeks of silence from the authors of a new paper, “Does Tweeting Improve Citations? One-Year Results from the TSSMN Prospective Randomized Trial” (Annals of Thoracic Surgery, 3 June 2020), I published a critique, pointing out various errors, omissions, and inconsistencies in their work. Keeping in character, the authors refused to respond, either with answers or with their dataset.
After much work, I was able to recreate their dataset from an appendix to an earlier work. Using their title list, I matched each paper to its citation performance in the Web of Science (Clarivate Analytics). I also looked up the article type for each paper on the participating journals’ websites. For the record, I am willing to share this dataset with anyone who asks.
Following the authors’ methods stated in their paper, I can report the following results:
First, the papers listed don’t match those described by their methods. The authors described selecting 112 “representative original scientific articles published from 2017-2018 in the Annals of Thoracic Surgery and The Journal of Thoracic and Cardiovascular Surgery.” The authors also explicitly excluded particular article types: “Articles that were not reporting on original scientific research were excluded, i.e. reviews, editorials, letters to the editor, and case reports.” However, their list of selected papers included 3 Reviews and 16 Editorials, one of which was entitled, “How Andre Agassi helped me during my cardiothoracic surgery training,” an inspiring personal note on how a tennis star helped the author persevere through his training. This paper was assigned to the Control group, as was another Editorial that was simultaneously published in 3 separate surgery journals . An author of this thrice-published Editorial was Mara Antonoff, the corresponding author of the tweeting study. Perhaps silence, in this case, was a much safer response than pleading ignorance.
Second, and more importantly, I detected no citation differences between the intervention (Tweeted) and Control arm of their study. Comparing citations accrued at a median of 861 days (2.4 years) from publication, Tweeted papers received 5.5 citations on average compared to 4.9 citations for the Control group — a difference of 0.6 citations (not significant, p=0.47). Removing the 16 Editorials and 3 Reviews from the dataset, the difference in citation performance gets even smaller: Tweeted (5.8) vs. Control (5.4), for a difference of just 0.4 citations (not significant, p=0.68), Figure below.
In their paper, the researchers reported a change of 3.1 citations for the tweetment group and 0.7 citations for their control group — a difference of 2.4 citations (p<0.001) in a period of just one year. They also reported that tweeting created a citation effect was 9.5 times the size of the control. It is difficult to believe how such a large effect could simply vanish.
Authorship comes with benefits and responsibilities: Both are necessary for the system of rewards in science to operate effectively. When authors are openly willing to derive benefits but accept no responsibilities, there is only one recourse: Editors must step in as arbiters and either compel the authors to clarify their claims or label their paper as untrustworthy. This case is understandably complicated, as both the first author (Jessica Luc) and corresponding author (Mara Antonoff) are both members of the journal’s editorial board.
But someone needs to speak up, even if deep conflicts of interest are at the root of this silence.
[An update to this investigation was published on 3 August 2020, Tweeting-Citations Authors Speak, Finally]
1. “Transatlantic Editorial: Thoracic Surgeons Need Recognition of Competence in Thoracic Oncology” was simultaneously published in The Annals of Thoracic Surgery, The Journal of Thoracic and Cardiovascular Surgery, and The European Journal of Cardio-Thoracic Surgery.