We seem, however, to be returning to the early 1970s where proposals for classifying the semantic intent of the citation were proposed and then abandoned in the realization of the cognitive work that would need to go into it, not to mention the fact that many citations are ambiguous, have multiple meanings and generally resist classification.

The main issue appears to be an attempt to create an empirical and quantitative system that resists subjective bias and intentional gaming.

Can this really be done? And more importantly, does it result in a better evaluative tool for scientists?

I agree that the problem seems to be how a person could go about ranking the merit of a citation in the context of another article. You gave an example where the four most cited articles came to vastly different conclusions than the un-cited articles on the same topic. Although in this scenario it is easy to see the discrepancies, I’m sure that we will run into problems down the road when the differences aren’t so obvious and the line isn’t easily drawn.

I like your idea of creating some kind of program to mark citations as “correct, diversionary, or invented.” It seems to be a very close cousin of Wikipedia’s method of notifying users when information is cited and sources are listed. The question is, how could you do this across all social sites and who would do it?

I think it’s really the responsibility of authors, reviewers, and editors to check references before publishing an article. Further, I think it’s the individual consumer’s responsibility to never assume the validity of sources on non-reviewed, social forums. It calls back to the old saying “Don’t believe everything you hear on TV.”

Even as I write this I realize the ideal nature of my suggestion and how relying on individuals to do the right thing has, thus far, left us swimming in specious facts.

I’m struggling on the issue of scalability though. Aren’t we essentially talking about a Faculty of 1000 for links – in which human intervention is needed to assess the merits of citations? And wouldn’t this introduce subjectivity throughout?

You suggest deploying social networks to eliminate this. But given the self-organizing nature of these and the preferential attachment (rich-get-richer) typically observed among nodes, isn’t there a danger the problem would be exacerbated? (Or do you envisage a publisher’s ‘invisible hand’ that validates, guides and prunes links?)