“If you are not paying for it, you’re not the customer; you’re the product being sold.”

So said Andrew Lewis aka blue beetle on a metafilter post about user driven discontent. I was thinking about this as I perused the latest Social Media (book marketing) YouTube video. And then, my Twitter feed coughed up an article from the New York Times entitled, “Got Twitter? You’ve been scored”

Here’s the opening paragraph:

IMAGINE a world in which we are assigned a number that indicates how influential we are. This number would help determine whether you receive a job, a hotel-room upgrade or free samples at the supermarket. If your influence score is low, you don’t get the promotion, the suite or the complimentary cookies. This is not science fiction. It’s happening to millions of social network users.

It sounds awfully familiar, doesn’t it? A single number for  “influence?”

It’s the impact factor, of course. As we all know, these numbers really matter — to the highly cited goes the grant funding, the tenure, and the fortune and glory (presumably).  They matter to publishers as well, of course. Here in the UK, these numbers have been used by universities as they chase money from the four higher education national funding councils — research from full-time members of staff has been filtered to select the “best” outputs (for which impact factor has been a well-used proxy). These are long-term decisions. The 2008 Research Assessment Exercise (RAE) results determined funding to universities through 2013. Interestingly (especially in the light of the Times article), the successor to the RAE, the catchily named Research Excellence Framework, will be using some nebulous concept of “impact” in order to dole out the cash.

Back to Twitter, Facebook, and the rest. As we move into the “Me Centered Web,” certain parties are very interested in working out how to make money off the free tools that have been provided for us to connect and share with. We are the product being sold. Companies like Klout and PeerIndex are attempting to distill the complex swirling interactions of the global social network into some simple numbers assigned to each user, whether we like it or not. I’m ranked based on my Twitter activity — I’ve not signed up to any of these companies, though note this: PeerIndex told me which of my followers had signed up.

As the Times article states, typically the 200 million plus users of Twitter, or the 750 million users of Facebook, are sorted into a numbering system that ranks them between 1 and 100. It seems a little crude even if you add in a touch of semantic analysis to the texts/links of the tweets, or the users’ biographies. These metrics also seem ripe to be gamed, especially when you take into account the (virtual) rewards based incentives the new companies are using to entice users back in order to improve the money-spinning database of influential profiles.

There are other issues as well. If you are not particularly active online (enter Clay Shirky or perhaps Malcolm Gladwell), you don’t rank for influence. Now, whatever one may think of Clay Shirky and his writing, the facts are that he’s very smart, very thoughtful, and what he writes is usually very important reading, especially in the area of social interactions and information creation in the digital age. A metric that can’t cope with the creator of two of the key works on the impact of the Internet on human information exchange might be deemed to be in need of improvement.

To be fair, the CEO of PeerIndex acknowledges the point. These influence metrics seem to have little room for acts of significant content creation. This is not a knee jerk “short form communications are rubbish” argument, but nevertheless measuring Facebook updates, link swaps, and likes or tweets does seem to largely exclude creators of longer form content. It’s tempting to point these new companies in the direction of the eigenfactor and to tell them to go take a look at the article level metrics of PLoS ONE or the work of project MESUR — it’s not often that scholarly publishing leads Silicon Valley.

But before we get too smug, the impact factor can also be gamed, and it has plenty of critics. There has been a steady stream of initiatives looking at different ways of measuring the influence of the researcher in their chosen field. Mendeley, altmetrics.org, academia.edu, PLoS ONE’s article metrics; all are tilting at the processes that keep our current quality metric in place. The Beyond Impact initiative is looking to try and synthesise a more complete view of what constitutes scholarly impact.

Buried in these initiatives is the idea that impact, influence, and quality will be determined by more complex signal processing than the impact factor provides for the content container we call the journal. Why? Because the things that need to be measured will extend out beyond the journal article. Don’t believe me? Go check out the Research Excellence Framework again, with it’s requirement to demonstrate the economic and social impacts of basic scholarly research.

Klout, PeerIndex, Facebook, Twitter, and the rest are all trying to solve the next problem in information filtration, and then the problem of how to monetise the solution.

Back in 1998, Sergey Brin and Larry Page effectively solved the filter failure issue of the early Web. PageRank was a signal:noise filter that transformed the cacophony of  the Web. They monetised by matching search terms to advertising, connecting users to companies touting their wares. But now, the next wave is upon us. You can’t argue with the numbers. Society, or at least a major subset of it, has embraced the next iteration of the Web — peer-to-peer information exchange at scale. And as this latest method of communication grows, we experience once again the familiar sensations resulting from being on the receiving end of too much information — sensory processing disorder on a global scale.

This problem will be solved. The company that does it will likely bring a transformation at least as dramatic as Google did.

Scholarly publishing should take no comfort from the fact that scholars don’t use Web 2.0 tools as part of their work. Some of them do. In fact, 13% of them do if you agree with the research. Some of those users like tilting at windmills. We shouldn’t dismiss Klout, PeerIndex, or whoever as just another bunch of VC funded upstarts. We should pay close attention. Social networks are not a fad. They are a fundamental component of the new information infrastructure. And we will all be analysed within them, one way or another. Publishers can have a meaningful role to play here. Just take a look at project cascade from the New York Times R&D department:

From long-form content to online reputation and influence. Now that’s how to solve a problem like Clay Shirky.