At last week’s Executive Colloquium hosted by Silverchair, Clay Shirky provided the keynote talk. In addition to detailing how the Dewey decimal system is failing us by locking us into information organization approaches that are both antiquated (in scale and world view) and ineffective, he also provided examples of how mathematicians, scientists, and others are solving problems in collaborative environments outside the traditional containers of journals and books.
One example he provided of how collaborative problem-solving can occur in our realm is a blog devoted to the P=NP problem by Dick Lipton, a professor of computer science at Georgia Tech. In the discussion highlighted by Shirky, an obscure mathematician had proposed on Lipton’s blog a proof that P≠NP.
Now, why do we care whether P=NP? Essentially, if every problem whose solution can be efficiently checked by a computer can also be efficiently solved by a computer, then it’s open season on computer security systems worldwide. Hackers the world over would rejoice, while e-commerce would screech to a halt.
A robust discussion and dissection of the supposed proof ensued on Dick Lipton’s blog. The proof underwent minor revisions, but endured for much longer than most. Hundreds of mathematicians contributed to the critique and improvement cycle in just a few days, speeding progress in ways traditional peer-review and publishing systems cannot do currently.
However, a problem remains, one which Shirky reflected on later during the Q&A session — namely, the lack of permanence and the lack of clear delineation of accountability both hamper these collaborative efforts, keeping them from the mainstream. This is the kind of problem that led to the housing market implosion, Shirky noted — small bits of information were mixed together outside robust systems meant to handle information transparently. In the mortgage market, very soon nobody knew what was in these collateralized debt obligations. Once the faith in these instruments evaporated, short-term credit markets underwent a shocking transformation, and soon they “broke the buck” — it suddenly cost more than a dollar to loan a dollar in the short-term cycles all businesses rely upon. This threatened to freeze credit markets, seize the economy, and create a widespread depression. Capital infusion was needed urgently and in large amounts.
Without clean lineages and a permanent, traceable record, science may make progress, but we might not know it or be able to exploit it. From an awareness perspective, those who need to know about advances may remain ignorant of them if advances are effectively hidden in fast-moving, peripheral systems. Worse, if claims later turn out to be wrong, we might not be able to unravel the resulting tangle. If enough science starts happening outside the system — from real scientists or charlatans — the faith in the system overall may falter.
The potential of faster, more collaborative science is huge. However, with traditional outlets remaining slow and deliberative to the point of leaving progress of any other kind unsupported, we are missing a huge opportunity. We are also creating huge liabilities. We need to create ways to support, clarify, and advance science even if it’s communicated in new ways, with new tools, and with new sources of input. If scientific collaboration is changing, why aren’t we?
The housing market imploded because the regulators were asleep at the wheel, and thought that new forms of financing were probably going to be governed by the overall market effects. Also, housing seemed immensely safe.
In the realm of scientific communication, publishers are part of the regulatory environment. And just because we’ve not imploded in the past doesn’t mean that a situation of scientific discourse and findings created outside our normal mechanisms won’t soon enough create problems for us that they effectively lead to our equivalent of the same place.
Remember, when the housing bubble popped, finance was the measure of what went wrong. But what really vanished was faith, trust, and confidence. If authors and readers lose faith, trust, and confidence in our ability to help them quickly advance their fields, we’ll have our own crisis of supply and demand.