Ratings correlates fairly poorly with every other metric. Combined with the low number of ratings, this makes me wonder if the option to rate papers on the journal web sites is all that useful.

We are likely to see lots of this sort of thing if commenting on papers becomes a regular practice.

We may be democratizing certain types of access, but we’re not democratizing attention. Just because we’re moving towards a state where anyone has the ability to get information into the stream does not mean that attention will be divided equally. Opening up access to the structures of distribution is not democratizing when distribution is no longer the organizing function.

Some in the room might immediately think, “Ah, but it’s a meritocracy. People will give their attention to what is best!” This too is mistaken logic. What people give their attention to depends on a whole set of factors that have nothing to do with what’s best….

People consume content that stimulates their mind and senses. That which angers, excites, energizes, entertains, or otherwise creates an emotional response. This is not always the “best” or most informative content, but that which triggers a reaction…

In a world of networked media, it’s easy to not get access to views from people who think from a different perspective. Information can and does flow in ways that create and reinforce social divides.

Perhaps someone like Thomson/ISI is indeed the way to go, a neutral third party could make the judgments without logrolling for their colleagues or propping up their own research. Of course, it would need to be transparent and consistent, easier said than done.

But I’d maintain that scientists’ opinions, especially those who have been around the block a time or two, are important and should be incorporated into the equation.

I’m just wondering if there’s a way to measure impact that doesn’t involve any sort of subjective judgment by an individual or group of individuals. Is there a way to take opinions and feelings out of the equation?

Our ‘folk’ are actually peer-selected, by the way. It’s community-driven: we just enable it. The other thing is that we’re not trying to rank the entire literature; rather we’re filtering it so that people can find the good stuff (as determined by their own community)–the rankings we have are more or less a by-product.