Good to know that your usage reflects what I hear from scientists, that reference management is still a higher priority for most than discovery. There’s no reason a site/software can’t serve both purposes well.

Great point about the snowball effect of a popularity breeding popularity. I touched on similar issues here.

I’d be willing to bet that any numbers from PLoS are skewed to an audience that specifically seeks out open acces/open source projects like CiteULike and preferentially uses them over commercial/closed projects.

—Isn’t the number of people who read and write peer-reveiwed literature around 6m worldwide?—

The NSF has numbers for the US here, their latest has something like 5.5 million working scientists in the US, with an additional 16.6 million working in “related fields”.

Thanks for your comments (and Euan), I hope I wasn’t overenthusiastic in my attempt at a “spirited” defense.

To address some of your points David, there is no doubt that the majority usage and benefit of the site is for personal online reference ‘Gathering’, not social or otherwise discovery. The bookmarklet makes it very quick and simple to save references online and that is why people use it. I

In fact I would go so far as to say if all it was a bookmarklet it would be used by many people.

I think the top papers, particularly in the all time list undoubtedly show a bias amongst our users for the subject. I don’t find this surprising.

“How to choose a good scientific problem” has obvious broad appeal, and if you read the comments it sounds like a great read.

What I also suspect is happening is a certain popularity breeding popularity effect that having a list like this engenders. Who knows.

However as you go down the list (the monthly and weekly lists are much more fluid) you see the science papers emerging. I agree, the big subjects seem to be the ones you mention.

Just to be clearer about the user numbers, that 300k is mainly non-registered users who come to the site from the web, search, links on publisher websites etc. (and look at more than one page). The number users who post stuff is much lower, as I said, in the 10s of thousands.

The 10k articles copied number is a count of how many times the copy?article URL is loaded (this URL is loaded when someone clicks the “copy” link on an article page) and is, I think, a good indication of a user copying an article into their library(ies). I have a reasonable arguments to think this is an undercount;

a) It doesn’t count the non-registered citeulike users who browse, discover stuff and go elsewhere,

b) The normal behaviour for our posting users may well be to follow the link to the publishers website, view/download the article and then post from there. That’s what I would do.

The last time I gave out that number was here: http://network.nature.com/people/mfenner/blog/2009/01/30/interview-with-kevin-emamy and I note that it was 6189, roughly a year ago. That’s good growth for the social stuff and is indicative of the real usage growth of the site as a whole.

(I just had a look and It’s doesn’t look like just a few users copying lots of articles, by the way).

At this point I’ll trot out some anecdotal evidence of people sharing stuff elsehwere:

@RafG I've followed your CiteULike feed for years now http://bit.ly/7kQ2zr – you like what I like and I've use you as my personal librarian.—
Marko A. Rodriguez (@twarko) January 04, 2010

http://friendfeed.com/kochlab/86405cb9/some-good-articles-in-this-person-citeulike

http://friendfeed.com/michaelnielsen/b13a98fb/citeulike-group-on-statistical-machine

http://www.slideshare.net/dullhunk/defrosting-the-digital-library-a-survey-of-bibliographic-tools-for-the-next-generation-web/53

As well as the data, by the way, one advantage of our public by default model is that you can browse the site yourself and get a feel for what is going on.

I repeat that my earlier assertion that the social bookmarking, everything public by default, 100% web based model has many advantages and has a firmly established place.

Where I take some issue with the publisher bookmarking systems is I think they are poorly executed and/or left unloved over time. You can look at the PLOS article level metrics data; a reasonable side by side comparision of the usage of citeulike vs. connotea and see that citeulike (at least for a PLOS audience) is about 5 times more popular. Thats a pretty big difference between two systems that are fundamentally doing the same thing.

As you have said elsewhere, it takes a certain scale for the social stuff to work, but I think that the success of the recommendation algorithms indicates strongly that citeulike has now got to a size where the dataset is genuinely useful in this regard.

Is it a niche activity? Science is a pretty niche activity. Isn’t the number of people who read and write peer-reveiwed literature around 6m worldwide?

It’s noteworthy, to me, reading some of your other blog posts, you are clearly well informed about and interested in this space, yet up to now you had fairly little exposure to citeulike. That’s my failing, but it does show that we have plenty of room to grow.

Although CiteULike doesn’t offer a summary of 2009, it’s interesting to me that the top paper in Mendeley is also the top paper in CiteULike’s last 28 day popularity listing (Uri Alon’s “How to choose a good scientific problem”). Given the differences in time scales, it’s hard to judge, though there are 3 papers on Mendeley’s top 10 that turn up in CiteULike’s all time top list, and 2 of them are on the last 28 days list. I wonder how much overlap there is between users of both sites. How many of Mendeley’s 100,000 users are active CiteULike users, listing the same papers on each service?

It’s also worth noting how many of CiteULike’s top papers are about doing science (how to pick a research problem, how to write a paper) rather than being actual data papers showing experimental results. Some of this is likely due to more general papers like this being popular across a wide spread of different types of scientists, but I think it’s also indicative that services like this are heavily used by the same Web 2.0 proponent type researchers I discussed here, people who are particularly interested in the way science is done, and who particularly like talking about it online.

It’s also telling how many of the top papers are in the same sorts of fields as seen in Mendeley, computational biology, systems biology, informatics and the like. Again, it shows that different fields have different cultures and are more or less comfortable with sharing/gathering information in this manner.

Kevin Emamy above lists the number of visits per month at around 300,000, and has around 10,000 articles copied back and forth between different users per month. The 10K number is hard to parse without further details, whether there are 10,000 users each copying one article from someone else, or 10 power users grabbing 1,000 articles each. But if this is a measure of how the site is being used for discovery, then at best, if each article was copied on a separate visit, then it accounts for what, 3% of the site’s use? How much of the site’s traffic is users uploading to their own account or looking up a paper they’ve already listed for themselves? This would give more of an indication whether the site is really being used for social bookmarking rather than for reference management.

I think Euan does raise some good questions as well, particularly on the growth rate of these various services. We’ve been told that it’s early days for these types of technologies and that scientists are a conservative lot. But five years does seem like a pretty long time, particularly considering something like Myspace launching in 2003 and experiencing it’s rise and fall by 2008 when Facebook passed it for membership numbers. Is this all part of a slow, steady climb, is this something that people are not aware of yet (even after all this time) or is this just a niche activity that simply has limited appeal?

I did. Except for citeulike of course

I hope NPG’s strategy wasn’t to replace Endnote with connotea! We encourage our users to use whatever desktop app they choose. And I don’t think that social bookmarking and reference management are the same thing.

As for comparing citeulike to other new solutions when you look at our citegiest posting data and compare it to others, citeulike holds up very well no? (for links See Mr. Crotty’s comment above)

Out of all the many “social” initiatives for scientists we’ve seen in recent years, citeulike has a strong claim to being a leader e.g.:
http://friendfeed.com/danielmietchen/c6254b29/quick-comparison-of-alexa-web-traffic-to-social

I’m excited that our recommender system (http://blog.citeulike.org/?p=11) has an 18% acceptance rate (ratio of accepted to rejected articles), and I’m not aware of any other live collaborative filter system for research papers. It works so well because of the data citeulike has collected.

Now the real challenge that you don’t mention yet we all face is how is it all going to be payed for? I must thank our sponsor Springer at this point.

Notwithstanding my earlier comments about not trying to replace reference managers, they are, it seems to me, institutionally mandated and payed for and really “enterprise software” models. If those decisions and budgets were put in the hands of the end users you would almost certainly see a different picture emerge today (I’m not advocating this because I don’t believe it will happen).

Where citeulike is really strong vs. it’s current competition is that it has managed to do all this with a ruthlessly low cost base. I think that’s the right strategy in this space right now.

5 years on, As Neil kindly says here: http://friendfeed.com/mfenner/ed934b88/are-publisher-linking-networks-like-2collab (we don’t pay people to say nice things) “CiteULike is alive, kicking and going from strength to strength.” I don’t call that resting on our laurels.

I find myself in the extremely uncomfortable position of wanting to quote Margaret Thatcher (“Turn if you want to” etc. ) but I’ll settle for Hunter S. Thompson instead: “Res ipsa loquitor”

> appeals to a significant cross section of scientists and academics

Without taking anything away from what CuL has accomplished so far – it’s a great system and innovative in lots of cool ways – it and Connotea have been going for almost five years. You’d have thought that if a truly significant number of academics were going to use social bookmarking regularly uptake would be better by now.

What do you think the ratio of active EndNote users to CiteULike (& Connotea, not singling out any one site) users is…? Do you think that ratio has changed in a noticeable way over the past four years?

What’s your current growth rate like compared to, say, Mendeley’s? Or Zotero’s?

IMHO the web based plays don’t give average users everthing they expect from a reference management solution (techy users who are OK with a certain amount of faffing are fine).

Not difficult to find evidence to back this up – am guessing that your users explicitly ask for desktop integration quite frequently… Connotea users certainly did a couple of years back.

Don’t want to sound negative, I think this space is tremendously exciting and that social bookmarking will genuinely improve the way researchers work but IMHO web based services need to quit resting on their laurels and raise their game.

… which is why spam filters may be just one of many changes that 2Collab and Connotea have to make to continue to be useful.

The total number of registered user accounts is 279,307, of those 58,545 are identified as spammers. The number of users who post stuff every month is in the 10s of thousands.
The Website receives around 900,000 unique visitors per month, of which 70% or so bounce (only visit one page) so the real number of browsers is 300,000 or so.

Search and tags are not mutually exclusive.

We do also have full text search of PDFs but see very few hits on that for some reason (I don’t think that functionality of the site is structured very well but that doesn’t explain the small number of PDF searches, IMHO.)

In the last 28 days citeulike users copied 9,859 articles directly from each other’s libraries. That’s the end result of all the social discovery.

I agree that the tagging is a pain for some, but it’s not required. We have search too.