At last week’s Executive Colloquium hosted by Silverchair, Clay Shirky provided the keynote talk. In addition to detailing how the Dewey decimal system is failing us by locking us into information organization approaches that are both antiquated (in scale and world view) and ineffective, he also provided examples of how mathematicians, scientists, and others are solving problems in collaborative environments outside the traditional containers of journals and books.
One example he provided of how collaborative problem-solving can occur in our realm is a blog devoted to the P=NP problem by Dick Lipton, a professor of computer science at Georgia Tech. In the discussion highlighted by Shirky, an obscure mathematician had proposed on Lipton’s blog a proof that P≠NP.
Now, why do we care whether P=NP? Essentially, if every problem whose solution can be efficiently checked by a computer can also be efficiently solved by a computer, then it’s open season on computer security systems worldwide. Hackers the world over would rejoice, while e-commerce would screech to a halt.
A robust discussion and dissection of the supposed proof ensued on Dick Lipton’s blog. The proof underwent minor revisions, but endured for much longer than most. Hundreds of mathematicians contributed to the critique and improvement cycle in just a few days, speeding progress in ways traditional peer-review and publishing systems cannot do currently.
However, a problem remains, one which Shirky reflected on later during the Q&A session — namely, the lack of permanence and the lack of clear delineation of accountability both hamper these collaborative efforts, keeping them from the mainstream. This is the kind of problem that led to the housing market implosion, Shirky noted — small bits of information were mixed together outside robust systems meant to handle information transparently. In the mortgage market, very soon nobody knew what was in these collateralized debt obligations. Once the faith in these instruments evaporated, short-term credit markets underwent a shocking transformation, and soon they “broke the buck” — it suddenly cost more than a dollar to loan a dollar in the short-term cycles all businesses rely upon. This threatened to freeze credit markets, seize the economy, and create a widespread depression. Capital infusion was needed urgently and in large amounts.
Without clean lineages and a permanent, traceable record, science may make progress, but we might not know it or be able to exploit it. From an awareness perspective, those who need to know about advances may remain ignorant of them if advances are effectively hidden in fast-moving, peripheral systems. Worse, if claims later turn out to be wrong, we might not be able to unravel the resulting tangle. If enough science starts happening outside the system — from real scientists or charlatans — the faith in the system overall may falter.
The potential of faster, more collaborative science is huge. However, with traditional outlets remaining slow and deliberative to the point of leaving progress of any other kind unsupported, we are missing a huge opportunity. We are also creating huge liabilities. We need to create ways to support, clarify, and advance science even if it’s communicated in new ways, with new tools, and with new sources of input. If scientific collaboration is changing, why aren’t we?
The housing market imploded because the regulators were asleep at the wheel, and thought that new forms of financing were probably going to be governed by the overall market effects. Also, housing seemed immensely safe.
In the realm of scientific communication, publishers are part of the regulatory environment. And just because we’ve not imploded in the past doesn’t mean that a situation of scientific discourse and findings created outside our normal mechanisms won’t soon enough create problems for us that they effectively lead to our equivalent of the same place.
Remember, when the housing bubble popped, finance was the measure of what went wrong. But what really vanished was faith, trust, and confidence. If authors and readers lose faith, trust, and confidence in our ability to help them quickly advance their fields, we’ll have our own crisis of supply and demand.
Discussion
4 Thoughts on "Permanence and Accountability — Why Publishers Need to Modernize Their Approaches"
The P = NP example is an interesting one and it does point out one of the biggest problems with social approaches and review of research. It was a high profile event, one that drew a lot of interest, interest that snowballed over time as more and more coverage came about. Which is great, but these sorts of systems tend to favor a lot of attention being paid to a small number of flashy glamor papers and no attention at all to the vast majority of articles, which are perhaps more niche, perhaps more pedestrian or perhaps just overlooked.
As shown by the article level metrics data that PLoS puts out, the vast majority of papers are destined to be unreviewed if “the crowd” is the only review method relied upon.
So while P = NP does raise some interesting theoretical questions, the robust discussion in engendered is more an outlier than the norm.
Yes, but I think the point is that without publishers directing audience to appropriate discussions like this, we do need to rely on the unreliable and unpredictable snowballs of interest you mention. If publishers were to take this kind of collaborative science seriously, then every field’s equivalent of “P=NP” might find its audience quickly, drive resolution of a question more ably than the current print-schedule-based system, and move science ahead faster.
As long as we support slow science, that’s what we’ll get. When we start to truly support more nimble communication among peers — and part of that is helping audience find materials and vice-versa — then we’ll seize the kind of opportunity foreshadowed by P=NP.
Most publishers indeed do take these sorts of discussions seriously and have tried desperately to encourage them (see every journal that accepts comments, letters to the editor, and big builds like the Nature Network). But as you note, the very nature of these sorts of discussions is unpredictable and researchers have resisted all attempts to formalize them.
Scientists in particular seem to avoid the comment threads of journal articles themselves because they are part of the permanent record. That’s almost the opposite of the problem you’re describing, people post using pseudonyms and don’t want to publicly be seen criticizing their peers (especially those sitting on their funding committees). Recording a discussion for posterity may prevent that discussion from ever happening.
There are also issues created by siloing of content by publishing houses. If the discussion won’t occur on the paper itself, then it’s much more likely to happen on a third party aggregator that brings together information from a variety of sources, rather than from a site sponsored by one publisher covering only the content they publish.
In the climate change area the system does seem to be imploding, and the reputation of science may be taking a hit. See for example. The semi-scientific blogosphere is largely responsible for this turmoil.
More generally, a lot of people in the US Government are looking at the emerging policy issues related to the kind of free-wheeling collaborative environment you describe. So is the scientific community. There were several sessions on this topic at this weekend’s AAAS annual meeting. A lot of people, including me, are developing new technologies to speed up scientific communication, but the potential for confusion is very real.