Classical Definition of Kno
Image via Wikipedia

Does scientific attention — as expressed through citations, media coverage, or practitioner knowledge — accrue to quality or reward the real contributors of breakthroughs? Or does attention in scientific publishing create a closed loop, raising questions of epistemology?

Recent posts suggest the closed loop, founder effects, and other problems emanating from an attention economy might be introducing detrimental effects to new breakthroughs, partly because there are too many old PhDs controlling the attention tools.

One reality of the attention economy in science is the Matthew Effect, named after a Biblical passage and popularized in 1968 by Robert K. Merton. Basically, it’s the “rich get richer” premise that once you start winning, you keep accruing benefits.

This is a well-studied phenomenon for citations. Once an article gets cited, it keeps getting cited. Once an article gets overlooked, it can disappear forever.

In fact, scholarship’s passive-aggressive culture ties right into the Matthew Effect — by doing nothing and ignoring research you don’t like or respect in hopes it will disappear, the poor get poorer, which only makes the rich relatively richer.

This process ultimately creates an imbalanced attention economy in scholarship, and two recent posts discuss some of the downsides of this, in addition to proposing some interesting solutions.

John Wilbanks discusses the Matthew Effect in a recent issue of SEED magazine, noting that:

. . . famous researchers have gathered the smartest and most ambitious graduate students and post-docs around them. . . . The famous grow more famous, and the younger researchers in their coterie are able to use that fame to their benefit. The effect of this concentration of power has finally trickled down to the level of funding: The average age on first receipt of the most common “starter” grants at the NIH is now almost 42. This means younger researchers without the strength of a fame-based community are cut out of the funding process, and their ideas, separate from an older researcher’s sphere of influence, don’t get pursued. This causes a founder effect in modern science, where the prestigious few dictate the direction of research. It’s not only unfair—it’s also actively dangerous to science’s progress.

Wilbanks believes that a merit system based on more than just the citation is now possible, and proposes a few ideas about how a merit system based on the usefulness of the resulting science, not just the data points of citations, could provide outlets for new ideas outside journal articles. Younger researchers could use these outlets to shine. Wilbanks’ approach would also probably offset the stranglehold older PhDs and researchers have, helping to create a more interesting terrain for knowledge.

While these are nice ideas, the funding issue Wilbanks refers to is the major force here, entrenching the attention economy firmly in the hands of older PhDs. As one PhD is quoted as saying in a post by Jason Hoyt on the blog for Mendeley, “it’s a Ponzi scheme.”

Well, it’s not a Ponzi scheme because attention can’t really run out, but it can be dominated. And if you dominate attention, you can dominate funding.

Trends in funding are creating frustration among the increasing number of young PhDs. These poor souls are adjusting to the funding drought by extending their time in the trenches — according to the National Science Foundation, 45% of all recent doctorates are now taking postdoc positions prior to a faculty appointment. This contrasts with only 31% following the same path 25 years ago. And postdoc positions are increasing in length of time as well, and are often followed by a second or even third “tour of duty.”

Grants By Age Group By Year

With attention resolving to senior researchers and older PhDs — along with the funds that follow — the number of new hypotheses or radical approaches may be diminishing. Funding the pet projects of people with citation slipstreams probably encourages people to draft behind the senior researchers, leading to more conservative and less adventuresome research.

Hoyt notes how grassroots efforts are trying to modulate these effects and break the stranglehold these researchers are sensing, with groups like the National Postdoctoral Association helping to spur policy changes at the NSF, NIH, and at numerous universities.

These tremors in PhD lifestyles, research funding, academic tracks, and recognition systems are worth watching. When money, reputations, and careers get involved, change seems likely to follow. A more dynamic, vibrant scientific community is at stake.

Reblog this post [with Zemanta]
Kent Anderson

Kent Anderson

Kent Anderson is the CEO of RedLink and RedLink Network, a past-President of SSP, and the founder of the Scholarly Kitchen. He has worked as Publisher at AAAS/Science, CEO/Publisher of JBJS, Inc., a publishing executive at the Massachusetts Medical Society, Publishing Director of the New England Journal of Medicine, and Director of Medical Journals at the American Academy of Pediatrics. Opinions on social media or blogs are his own.


30 Thoughts on "Old PhDs and the Matthew Effect — Is the Attention Economy of Scholarship Making Science Too Staid?"

There may be a cause versus effect issue here. Many people regard the rise of postdoctoral education as a good thing, not a refuge. This increase in turn shifts the grant getting curve to the older side, but the younger researchers are getting funded too, as postdocs.

Beyond that you seem to be claiming that younger researchers have better ideas than older ones, which is questionable, to say the least. Nor do program officers fund proposals based on citation counts, although there is a sensible need to continue funding successful programs. In short, there is a lot of mythology in this issue, some of which traces back to Merton.

I think the point isn’t about “better” ideas but about “different” ideas. Scientific breakthroughs tend to occur when young, trailblazing researchers are allowed to pursue novel approaches to knotty problems. By creating an attention slipstream consisting of citations and funding, the draw of that slipstream may prevent many from veering off to pursue new directions. As we all know, ideas that were originally dismissed have time and again come to be seen as groundbreaking and game-changing. “Experts” are a notoriously conservative lot. Who knows what we’re missing out on?

I do not understand your distinction between better and different ideas. The vast majority of ideas that are originally dismissed (as merely different, but no better) stay dismissed, and for good reason. The fact that a few truly great ideas were not accepted at first is no reason to fund a lot of young people to the detriment of established research programs. What specifically are you proposing?

Do you have any data showing the age distribution for breakthrough ideas? I have not seen that. We here in the science of science community would be very interested in such data. Perhaps you are confusing a researcher’s first good idea, which starts their research program, with all the good ideas that come after. I had a breakthrough 2 years ago, at age 65.

I don’t understand the “slipstream” metaphor.

I think Kent is getting at a long-held philosophical belief, covered in this Jonah Lehrer article that states:

Modern science is populated by expert insiders, schooled in narrow disciplines. Researchers have all studied the same thick textbooks, which make the world of fact seem settled. This led Kuhn, the philosopher of science, to argue that the only scientists capable of acknowledging the anomalies — and thus shifting paradigms and starting revolutions — are “either very young or very new to the field.” In other words, they are classic outsiders, naive and untenured. They aren’t inhibited from noticing the failures that point toward new possibilities.

Philosophical beliefs (or what I referred to as mythology) are no basis for science policy. I myself am a Kuhnian, but you can’t buy more paradigm shifts by funding more young researchers (at the expense of old researchers).

Nor do we necessarily want more paradigm shifts because, as Kuhn pointed out, productive science occurs under stable paradigms. Perpetual revolution is a prescription for intellectual bankruptcy. In fact it is impossible. That experts are the problem is a common fallacy.

I’ll elaborate. A better idea is an improvement on a current paradigm, sort of like a better mousetrap. A different idea is perhaps a new way to control mice, like mouse birth control or sonic repellents. “Better” ideas are incremental improvements along current lines of thinking. “Different” ideas show new routes to accomplishing a goal or reveal discoveries that couldn’t be seen from the previous path. These are breakthroughs.

You are young at heart, that’s great. I’ll have to dig for the data on the age of a researcher’s first good idea. If anyone else has it handy, please link to it in a comment.

A “slipstream” is the aerodynamic effect accomplished when an object moving ahead creates turbulence sufficient that a following object meets less wind resistance or is even sucked along by the relative lower pressure. Slipstreams let race car drivers, cyclists, and runners conserve energy by closely sharing the path of the object ahead. My intent in using the metaphor was to underscore the notion that the attention economy is creating these larger objects in front of younger researchers, lowering the pressure on them, making it hard for them to see ahead, and encouraging them to coast.

The age of a researcher’s first good idea is irrelevant, so apparently you did not understand me. Your claim seems to be that young researchers have most of the breakthroughs, which I dispute. This is strong scientific claim.

I also dispute your apparent claim that postdocs are coasting, as opposed to learning and doing research. Are you proposing a policy against postdoctoral education?

How does the increasing number of Ph.D.’s granted factor into this? I can’t find the NSF numbers for 25 years ago that you mention above, but looking at the 1995 numbers here versus 2006, they list 542,540 doctorates granted in 1995, 711, 800 total in 2006 (for science only the numbers are 455,530 and 561,230). That means over those 11 years degrees granted increased by 32% (or 23% for science-only). If the number of people taking postdocs before faculty positions increased by only 14% (and that’s over 25 years), doesn’t that mean that there are many more faculty opportunities available, but that they just haven’t kept pace with the increase in degrees granted?

I think the concept that these figures allude to is Agenda Setting and not the Matthew Effect.

Agenda Setting comes from mass media studies of the news and the power of large presses (e.g. The New York Times) to determine what is important (or newsworthy) and therefore sets the national debate.

Those who decide how research grants are allocated have similar power — those who receive funds are permitted to generate new knowledge, and subsequently, to speak to the rest of the research community. Deny funds to a researcher and you have essentially silenced him.

I agree that a certain amount of framing occurs, so they’re related. And, as you note, Agenda Setting is where the rubber meets the road, so that concept is much more pointed than the softer Matthew Effect. Nice addition!

Fascinating discussion. I’d like to point out that the fact that social media isn’t dominated by the Elder Statesmen of Science argues for the value of social media in science, as it provides a secondary attention market in which new young leaders can emerge. Novel attention and influence metrics, such as those being developed by Mendeley (thanks for the plug above) can strengthen and formalize this secondary market, promoting innovation from exactly the place the old guard least expects it – the frivolity of social media.

That’s a very interesting study, Kent, and it backs up what people like Cameron Neylon, Jason Hoyt, and I have been saying for some time now – that the “Zeitgeist of “build it and they will come”” doesn’t work in practice.

My argument above isn’t pre-supposing heavy involvement of scientists in social media. Rather, I’m saying that if more young academics were to get involved, they’d have a better chance at really doing something innovative because they’d have a larger share in the secondary attention market.

I’m quite aware that adoption of new collaboration tools isn’t where it should be, but engagement is growing on a daily basis. For example, the aggregate number of documents added to Mendeley libraries is approaching the total number of documents in Pubmed, but it hasn’t been slow linear growth. It’s very much been a “hockey stick” kind of growth, so it’s not surprising to me that a 5-year study including 160 interviews, sourced through traditional channels, would miss some of these larger trends that incubated outside of the elite circles from which the study did its “snowball sampling”.

Now, there’s a lot packed into what I’ve said above, and we could talk for hours about any one of the assertions I’ve made, but I think it may be most useful to ask, “If ‘build it and they will come’ doesn’t work, what does?” Here’s what we who correctly identified that that model wouldn’t work years ago think will work:

Add value up front and let networks emerge from the latent connections in the data. This was pithily summed up by Jon Udell in his summary of the idea from Jeff Jonas, when he said “data finds data, then people find people.” Cameron has related how this idea works within the realm of science in a recent post about Sciencefeed and and why it’s not quite there yet.

I’d love to know what you think about this idea, Kent, as well as what the larger community of academics here think about the viability of the secondary attention market.

My problem is that I work in the field of the science of science policy. For example: We have this idea that science policy should be based on factual analysis, not myths, metaphors and wishes. Shutting down post-docs or shifting funds to young investigators is a major policy issue.

So I thought of a way that we could maybe begin to measure the age distribution of breakthroughs. How about this indicator? Any paper with 100 or more citations after 2 years was reporting a breakthrough. It is possible to take a sample of such papers and identify the age of the lead author of each at the time of publication. This gives us an age distribution. We can even normalize it based on the total number of people publishing at each age, to get the percentage of breakthroughs by age.

Does this work as a basic approach? It would be an interesting demographic. NSF might even pay for it.

Isn’t it more complex than that? If you’re in your early 20’s and make a big breakthrough (and are listed as the lead author on the paper), you’re probably a graduate student in someone else’s lab. The funding that led to that breakthrough was given to the PI of that lab, not the young student. So aren’t you actually funding the young researchers who make breakthroughs when you fund an older PI?

There are several issues here. The demographic issue is Kent’s questionable claim that young researchers have more breakthroughs than experienced ones. Funding is irrelevant.

There then arises his equally contentious claim that young researchers that work for somebody else are thereby prevented from having as many breakthroughs. That would take a different study, probably a comparison between post-docs and independent researches of the same age or experience.

Teasing out who has breakthroughs in a system like the one we have is difficult, especially if funding is making older researchers more able to dictate the agenda.

Funding is not irrelevant. If I have funding, I can tell my advisor to pound sand and pursue my own agenda. If I’m reliant on a PI, I’ll be more likely to toe the line. Funding is precisely the relevant piece.

Are you suggesting that graduate students be given the same sized grants as a PI running a lab? What overhead does a graduate student have in an already established lab? There are lots of fellowships available for students and postdocs that provide a great degree of intellectual freedom even when working for someone else. Those don’t seem to be accounted for in your stats above, just looking at people starting their own labs. Has this sort of funding increased or decreased in recent years?

I am trying to break the problem down into separately testable hypotheses. That is how science works. We can confine the initial breakthrough demographics analysis to those who have funding, making funding irrelevant as a parameter.

This is just restating the problem as a non-problem — i.e., if a young researcher is part of an older PhD’s lab and gets indirect funding thanks to that, then it’s all OK, and the attention economy accrues accordingly. The lead author position isn’t guaranteed, and funding is dispersed through a lab, not given directly to the young hotshot. So, is it still a non-problem?

Shifts like that shown in the graph are significant, and as Phil pointed out in a separate comment, the notion of Agenda Setting isn’t to be ignored. Who knows how many ideas are squelched because the funding recipient has his/her own agenda.

Most labs I know welcome postdocs with their own funding sources and grant them a great deal of freedom to pursue that funded research. There are many direct fellowships for positions like this (and for grad students as well) that aren’t “dispersed through a lab”. A graduate student probably doesn’t need an RO1 grant to accomplish their project.

The chart was % of grants, not % of $$. Younger researchers are getting a lower % of grants available. That seems a potential problem, possibly fed by the attention economy of science, and fueling more agenda-setting by senior researchers with invested stances in current theoretical frameworks.

It’s hard to tell, in a vacuum though. What exactly does “grants” encompass? Plus it pays to look at the total numbers, not just the percentages. How many Ph.D.’s are being granted year to year? What age are Ph.D. recipients (is graduate school taking longer or shorter than it used to)? How many PI’s are starting their own labs before the age of 25 anyway? Are there more grants than ever for young investigators, but more young investigators than ever, so even though the opportunities have increased, the percentages have dropped? What is the context for the number you used?

How many ideas are squelched is what I am trying to estimate. It is called science. If there is no evidence for your claims then there is no basis to change anything.

Also, should a multi-million dollar grant be trusted to someone who has yet to prove they can finish their degree?

That’s an interesting idea, David. Please feel free to email me, I’d like to help in any way I can.

I help run a research program on how to accelerate science by improving communication. We have never considered demographics so this is an interesting issue for us. We have considered post-docs to be an important mechanism for the transmission of existing ideas, not as an obstacle to new ideas. There may be an important trade-off here, between communication and creativity. Progress requires both.

I have however experienced the creativity problem. In my Ph.D. thesis I got results that suggested that what my committee members taught was wrong. This was not a result they were willing to accept so I almost did not get the degree. In the end they only let me through because the liked me. But this does not imply that we want fewer post-docs.

Comments are closed.