Thanks for discussing this interesting paper. Given the ‘light’ refereeing process for contributed papers in PNAS, it is to be expected that they have lower citation rates on average. Of course academy members are producing high quality papers but their incentive is to send the best of them to Nature, Science and Cell while sending papers that are more difficult to publish to PNAS as contributed papers.

In my paper with Nicolas Maystre where we analyzed the effect of open access in sample of PNAS papers ( http://ideas.repec.org/p/cmi/wpaper/cemi-workingpaper-2008-007.html ; http://scholarlykitchen.sspnet.org/2008/11/18/author-pays-oa/ ) we also had the result that papers that were directly submitted received more citations (controlling for a number of factors including past productivity of the last author) which we did not investigate further.

What is really interesting is the finding that the most cited papers in PNAS appear to be contributed, rather than directly submitted. I interpret this result as suggesting that academy members sometimes use the contribution track to publish very important papers quickly, possibly bypassing a conservative bias in the top three journals.

Patrick Gaule

My main concern comes down to whether you may be attributing citation differences to the peer-review process when they may be explained by other variables, such as author effects.

Several secondary analyses could have been conducted to rule out some of the competing hypotheses, for instance:

1. Comparing NAS member articles that went through Communicated versus Direct submission tracks would allow you to understand the effect of self-selection.

2. Using the number of authors as a variable in your regression model would help to control for self-citation.

3. Analyzing the top and bottom 10% for more clues of causation such as the age of the author (remember that NAS membership is for life), or funding source.

You may be completely right that peer-review is responsible for your findings, but without ruling out competing explanations, you may be reporting on a spurious association.

Thanks very much for your thoughtful posting about our paper.

I completely agree that author effects are an important issue which we were not able to include in our analysis because we did not have the data readily available.

However, all papers published through the Contributed track have NAS-member authors. Thus if prestige of author was responsible for increased citation rates, we would expect to see higher citation counts across all Contributed papers. However, we do not. Instead Contributed papers are cited less on average, and only cited more in the extreme. This makes me believe that the success of the very successful Contributed papers is not driven by author prestige.

We also agree that the issue of selection bias in which papers get sent to which track is an important issue, and discuss this our conclusion:
“In addition to the differences in referee selection and review process, [NAS member] authors may choose to submit papers they feel are stronger or weaker through particular tracks. The direction of this effect, however, is unclear. For example, one could hypothesize that weaker papers are submitted through alternative tracks to increase the probability of acceptance, or that stronger papers are submitted through alternative tracks to increase the speed of acceptance. Or perhaps both suggestions are correct. Exploring this issue also merits future study.”

I’d love to hear your thoughts!

Thanks,

David G. Rand
Harvard University
drand@fas.harvard.edu
http://www.DavidGertlerRand.com