Peer review. Image via Wikipedia
Is peer review in decline? Glenn Ellison, an economist at MIT, is beginning to question the added value of being published in top journals, at least for high-ranking authors.
Ellison has painstakingly documented the decline of articles published in top economics journals by authors working in the highest-ranked schools. These authors are continuing to publish, but are seeking other outlets, including unrefereed preprint and working paper servers.
There are several explanations for this trend, and Ellison is careful about not attributing what he has observed to a single theory. Still, there are two explanations that make empirical and theoretical sense:
- The Internet has allowed the certification and dissemination functions of journals to be disaggregated, permitting other services (like preprint and working paper servers), and networked search tools (like Google Scholar), to perform the function of disseminating research findings.
- Any economist will tell you that it is taking more time to get your work published in a top-economics journal. Submission to publication may take years in many cases, and reviewers are more eager to require multiple revisions from the authors. For those economists who have already built a reputation, the benefits of going through the certification process may not be worth the effort, at least not for all of their work.
Ellison writes:
More top economists may realize that the publication hassles they have been enduring are not necessary. The peer-review process may also be subject to unraveling: as more top economists withdraw from the process, the signal that publication in a given journal provides is devalued, and this may lead to further withdrawals.
Since the prestige of a journal is heavily influenced by a small number of influential papers, losing these contributions can be significant in a publishing environment where authors want their work associated with other high-profile authors. This is what Ellison means by “unraveling,” a negative feedback loop in which market confidence is eroded. Think collapse on Wall Street, only much slower.
Ellison concludes with a warning on what such a decline of peer review could mean, especially if no alternative certification system is ready to step in.
One could imagine that new institutions may arise and perform many of the same functions as the current peer-review system more efficiently. Given how central peer-review has been to academic research over the past century, however, the thought that the current system might collapse before any successor is clearly established is troubling.
It may not be surprising that Ellison’s paper is not published in a journal, but resides in two online repositories (SSRN and NBER), not to mention Ellison’s own home page.
![Reblog this post [with Zemanta]](http://img.zemanta.com/reblog_e.png?x-id=4e7e26b3-ae0f-4226-9a88-797a2b58a2e8)
Oct 7, 2008 at 5:39 pm
This sounds similar to the experiences of physics and mathematics authors about 15 years ago. Publication often took more than a year after acceptance. Much work was considered outdated by the time it appeared in a journal. Authors circulated their papers on the LANL server, which extended online a long tradition of circulating preprints and provided informal peer review. Some thought (and hoped) the LANL server would be the end of physics journals. Instead, it seemed to improve their content because of the useful comments authors received. However, authors still considered publication in a traditional journal important, even after lengthy delays.
Oct 8, 2008 at 7:47 pm
[...] Review Collapse Posted on 9 October 2008 by Michael Philip David in his Scholarly Kitchen post asks the important question ‘Is peer review in decline?”. He cites Glenn Ellison, an economist [...]
Oct 9, 2008 at 2:24 pm
Ellison has also helpfully self-archived his paper on his homepage (http://econ-www.mit.edu/files/906). The SSRN and NBER repositories, while not “publications” per se, still charge for access.
Oct 10, 2008 at 6:53 am
[...] The Collapse of Peer Review « The Scholarly Kitchen [...]
Oct 10, 2008 at 10:06 am
[...] work on the decline of peer review (reported earlier on my blog) is still being discussed: Ellison has painstakingly documented the decline of articles published in top economics journals [...]
Nov 21, 2008 at 11:41 pm
I had an interesting conversation with Charles Lowry (Director at ARL) about this topic. Sure, senior-level faculty don’t have to publish in top-tier journals. They already have the credibility that a journal would normally provide. The benefits of working with an established publisher are lessoned, when you don’t need the benefits of certification, review, or marketing, which are the three key value-add services that publishers provide. For example, I would likely read Thomas Friedman, if he didn’t publish in the NY Times. However, tenure review boards still rely heavily on the perceived quality of the journals where a junior faculty has published, possibly as much as they consider the work actually produced.
If the journal world is slowly eroding, as it seems it is, and if scholarship moves to publication in repositories of various types, new methods of assessing the quality of content will need to develop, which more closely assess a single work, rather than the packaging (in this current case, the journal). This will need to happen for two reasons: the first being the assessment and review process, particularly tied to tenure. The second is tied to the need of scholars to have some shorthand view of what is critical for them to read and be aware of in their field(s).
Nov 23, 2008 at 1:14 pm
In the life sciences, we’re desperately wishing the shiny-paper journals would go away.
Senior researchers don’t want to publish there, but feel pressured to help their trainees get jobs by publishing there.
Trainees don’t want to publish there, but feel pressured to publish there if we want jobs.
A negative feedback loop is exactly what we need to get rid of it, and switch to other possibilities (like wiki-type review online).
Unfortunately, it’s hard to see how we can break the cycle, unless senior researchers could be induced to stop including themselves on author lists (even when they have contributed next to nothing but their reputations).
I’d rather see a shift to junior researchers publishing independently, acknowledging but their advisors but not giving them all the credit. Then maybe the pressure to publish in certain journals would lessen if the job market would learn to value different qualities in applicants.