As an institution, science is not fond of privilege. Success in science is supposed to be the result of merit — hard work, tenacity and, to some degree, sheer luck — not nepotism, favoritism, or entitlement. At the same time, we like to reward successful scientists with prizes and memberships to elite academies, and along with these rewards often comes certain privileges. One such privilege is accelerated publication.
Based on the National Academy of Sciences model of contributed submission, the journal, mBio, allows a different submission pathway for fellows of the American Academy of Microbiology (AAM), the honorific leadership group within the American Society for Microbiology (ASM). Fellows are permitted to select their own reviewers and submit their paper (along with its reviews) directly to the editor. The editor retains the right to reject the submission, require a new round of external review or ask for extensive revisions. For most contributed submissions, papers are published much faster–often within days–than the normal mBio publication route.
Is this VIP track for academy fellows beneficial to the performance of mBio? One could surmise that submissions from fellows of an elite academy of researchers would perform better than submissions from non-membered authors. On the other hand, one could equally argue that by allowing academy fellows to chose their own reviewers ensures a less-rigorous review process and allows sub-standard work to slip through to publication, a longtime criticism of PNAS’s contributed track.
How do papers from these two mBio author groups perform? Based on the raw distribution of citations, AAM-fellow papers were clear winners, receiving nearly twice as many citations as non-fellow papers (Figure 1). A non-parametric (Wilcoxon) statistical test confirms this difference. Even if you are not big on significance testing, simply plot the data and eyeball the distribution. The difference is big.
There is an important variable that we miss when we simply compare these two publication groups; however, and that is time. Papers start accumulating citations months after publication and increase their numbers over time in a non-linear fashion. By ignoring time as a confounding variable, we may attribute the citation differences to AAM fellowship, while the real cause may be something else.
If we plot papers by fellowship status over time (Figure 2), you’ll notice that mBio papers contributed by AAM fellows (blue) have been relatively stable with some initial growth when the journal began publication in 2010. In contrast, papers published by non-fellows (grey) have been growing steadily over time, meaning that they are younger, on average, than AAM papers.
Control for the age of publication of mBio papers and the large citation effect we observed completely disappears (Figure 3). If we draw two best-fit regression lines through the data, both lines fall on top of each other. Time, to paraphrase musician Gillian Welch, was the Revelator.
In their editorial, Arturo Casadevall, Editor-in-Chief of mBio and Tom Shenk, Chair of the Journals Board of the ASM argue that the results suggest no overt bias in the publication process. AAM Fellows are not rewarded for a free publication ride in mBio. The quality in these two groups of papers appears to be equal.
So, what does this study tell us about peer review? First, I don’t think we can make any grand claims about reviewer selection, blind vs. unblinded, or editorial process without putting mBio’s submission track for AAM Fellows in context. As Casadevall and Shenk explain, both AAM Fellows and their reviewers must attest that these papers represent the top 10% of their field–a similar determination made for reviewers evaluating non-fellow submissions. And about 15% of papers submitted through the contributed track are rejected for publication by the handling editor, meaning that having AAM fellowship and supportive reviewers are not sufficient conditions for publication. The only strong claim mBio can make for their contributed papers track is speed to publication. Yet, in a competitive journal market for top papers, speed may be good enough.
7 Thoughts on "Do Academy Members Publish Better Papers?"
There is one other factor and that is many search for articles based on citations. Thus the impact of citation may determine whether or not a paper that does cite benefits by citing the article. Thus it further ranks the cited paper while it gets the citing paper on the radar whether or not the cited paper is significant or of significant inherent value to the citing paper. It’s where the citation indices gained market at one time.
Looking at the citations of AAM fellows vs non fellows, do you have any idea why the immediacy of the citations from AAM fellows is increased?
By “immediacy of citations” do you mean measuring whether AAM-fellow papers are cited earlier than non-AMA-fellow papers? I didn’t test citation rate in this study, but if you have the data and measured this, would you respond and let us know what you found? If I misunderstood your question, would you clarify? Thanks!
Sorry for the confusion. I meant in the figure 2, do you know why the AAM-Fellow papers were more cited from 2010-2012 than non AAM-Felows papers? Thanks !
Sorry, I’m still confused. Figure 2 shows publication frequency for both tracks over time, not citations. According to the EiC (Arturo Casadevall), submissions of AAM track papers grew in the first two years. And because AAM submissions have a high acceptance rate, submission growth resulted in publication growth. As for regular submissions, he claims that submissions have been the result of a strong mBio Impact Factor. If I still don’t understand your question, please clarify. Thx.