Comments on: The Impact Factor: A Tool from a Bygone Era? http://scholarlykitchen.sspnet.org/2009/06/29/is-the-impact-factor-from-a-bygone-era/ What's Hot & What's Cooking in Scholarly Publishing - from the Society for Scholarly Publishing Fri, 12 Mar 2010 00:49:33 +0000 http://wordpress.com/ hourly 1 By: Peter Binfield http://scholarlykitchen.sspnet.org/2009/06/29/is-the-impact-factor-from-a-bygone-era/#comment-4059 Peter Binfield Tue, 28 Jul 2009 15:32:22 +0000 http://scholarlykitchen.sspnet.org/?p=4699#comment-4059 Torsten - you are thinking about the journal in a somewhat traditional manner though I think. Why should the quality of an article be a binary thing? (i.e. "it is in Nature, therefore it is good. vs. It is not in Nature, therefore it is bad"). An alternative could be to publish everything that passes peer review but then have the peer reviewers and or editors assign a 'grade' to each paper. In this way, the grade could be used as a "day one" indicator of possible value of the article, as you requested (and it would have been measured by the same system which is currently used to make accept/reject decisions on a selective journal) and this group of "highly graded" papers could then become a high quality brand of their own. Torsten – you are thinking about the journal in a somewhat traditional manner though I think.

Why should the quality of an article be a binary thing? (i.e. “it is in Nature, therefore it is good. vs. It is not in Nature, therefore it is bad”). An alternative could be to publish everything that passes peer review but then have the peer reviewers and or editors assign a ‘grade’ to each paper. In this way, the grade could be used as a “day one” indicator of possible value of the article, as you requested (and it would have been measured by the same system which is currently used to make accept/reject decisions on a selective journal) and this group of “highly graded” papers could then become a high quality brand of their own.

]]> By: Kent Anderson http://scholarlykitchen.sspnet.org/2009/06/29/is-the-impact-factor-from-a-bygone-era/#comment-4058 Kent Anderson Tue, 28 Jul 2009 14:54:12 +0000 http://scholarlykitchen.sspnet.org/?p=4699#comment-4058 Blog coverage and downloads occur much more quickly than citations. The issue you're getting at is brand power. We assume that the most important papers find their way into the most important journals. There is a strong correlation, but cause-effect is a bit harder to ferret out. Also, the utility of a paper may be hidden by this. Many clinical/practitioner/bench journals publish useful studies that aren't cited because they're used instead. With the current impact factor system, citations take months or years to accumulate. We live in a faster, more fluid information environment than the one Eugene Garfield predicated his system upon. Blog coverage and downloads occur much more quickly than citations. The issue you’re getting at is brand power. We assume that the most important papers find their way into the most important journals. There is a strong correlation, but cause-effect is a bit harder to ferret out. Also, the utility of a paper may be hidden by this. Many clinical/practitioner/bench journals publish useful studies that aren’t cited because they’re used instead. With the current impact factor system, citations take months or years to accumulate. We live in a faster, more fluid information environment than the one Eugene Garfield predicated his system upon.

]]> By: Torsten http://scholarlykitchen.sspnet.org/2009/06/29/is-the-impact-factor-from-a-bygone-era/#comment-4057 Torsten Tue, 28 Jul 2009 10:14:39 +0000 http://scholarlykitchen.sspnet.org/?p=4699#comment-4057 If one needs all this data like blog coverage and downloads to see the value of an article does that not mean that I need to wait some month after the article came out to see if it is a good one? If a paper appears in Nature etc. I would assume it is worth reading right away. The same problem appears if I'm a young scientist applying for my first grant and all of my papers have just been published recently. How could I convince people that the research I did has a gread impact if this will only be seen some month/years later? If one needs all this data like blog coverage and downloads to see the value of an article does that not mean that I need to wait some month after the article came out to see if it is a good one? If a paper appears in Nature etc. I would assume it is worth reading right away. The same problem appears if I’m a young scientist applying for my first grant and all of my papers have just been published recently. How could I convince people that the research I did has a gread impact if this will only be seen some month/years later?

]]> By: PLoS Journals – measuring impact where it matters « Speaking of Medicine http://scholarlykitchen.sspnet.org/2009/06/29/is-the-impact-factor-from-a-bygone-era/#comment-3982 PLoS Journals – measuring impact where it matters « Speaking of Medicine Wed, 22 Jul 2009 20:20:02 +0000 http://scholarlykitchen.sspnet.org/?p=4699#comment-3982 [...] As summarized by Michael Jensen, and discussed by many others including recently over at the Scholarly Kitchen, there’s a lot more to scientific impact than citations in the selection of journals covered by [...] [...] As summarized by Michael Jensen, and discussed by many others including recently over at the Scholarly Kitchen, there’s a lot more to scientific impact than citations in the selection of journals covered by [...]

]]> By: PLoS Journals – measuring impact where it matters « everyONE – the PLoS ONE community blog http://scholarlykitchen.sspnet.org/2009/06/29/is-the-impact-factor-from-a-bygone-era/#comment-3979 PLoS Journals – measuring impact where it matters « everyONE – the PLoS ONE community blog Wed, 22 Jul 2009 19:26:39 +0000 http://scholarlykitchen.sspnet.org/?p=4699#comment-3979 [...] As summarized by Michael Jensen, and discussed by many others including recently over at the Scholarly Kitchen, there’s a lot more to scientific impact than citations in the selection of journals covered by [...] [...] As summarized by Michael Jensen, and discussed by many others including recently over at the Scholarly Kitchen, there’s a lot more to scientific impact than citations in the selection of journals covered by [...]

]]> By: memotypic http://scholarlykitchen.sspnet.org/2009/06/29/is-the-impact-factor-from-a-bygone-era/#comment-3768 memotypic Fri, 03 Jul 2009 12:38:49 +0000 http://scholarlykitchen.sspnet.org/?p=4699#comment-3768 This is interesting for that: http://thedata.org/ and there are lots of discussions and other things appearing, but it has to be more or less universal (DOIs as a component would help, maybe with OpenIDs, in some kind of registry). This is interesting for that: http://thedata.org/ and there are lots of discussions and other things appearing, but it has to be more or less universal (DOIs as a component would help, maybe with OpenIDs, in some kind of registry).

]]> By: memotypic http://scholarlykitchen.sspnet.org/2009/06/29/is-the-impact-factor-from-a-bygone-era/#comment-3767 memotypic Fri, 03 Jul 2009 12:33:10 +0000 http://scholarlykitchen.sspnet.org/?p=4699#comment-3767 Going on a slight tangent, I'd like IF to be broadened to include use of data sets via citations of somethin like a DOI attached when shared (by a vestigial journal that only asks for a form with community-specified minimum info in, and ultimately direct from databases). I'd also like a universal person ID (for citation purposes) at which to bank all such citations (and in passing to do away with this issue: http://tinyurl.com/aqtpbn) and maybe even to bank other kinds of credit (time spent reviewing? training?) Going on a slight tangent, I’d like IF to be broadened to include use of data sets via citations of somethin like a DOI attached when shared (by a vestigial journal that only asks for a form with community-specified minimum info in, and ultimately direct from databases). I’d also like a universal person ID (for citation purposes) at which to bank all such citations (and in passing to do away with this issue: http://tinyurl.com/aqtpbn) and maybe even to bank other kinds of credit (time spent reviewing? training?)

]]> By: Susan http://scholarlykitchen.sspnet.org/2009/06/29/is-the-impact-factor-from-a-bygone-era/#comment-3764 Susan Fri, 03 Jul 2009 09:15:49 +0000 http://scholarlykitchen.sspnet.org/?p=4699#comment-3764 I would add to Jo's comment that Impact Factor only takes into consideration the journals indexed by Thomson which is a tiny subset of the the scholarly literature. Dose this mean that if you are not indexed in the Web of Knowledge, you don't exist? I would add to Jo’s comment that Impact Factor only takes into consideration the journals indexed by Thomson which is a tiny subset of the the scholarly literature. Dose this mean that if you are not indexed in the Web of Knowledge, you don’t exist?

]]> By: Bev Acreman http://scholarlykitchen.sspnet.org/2009/06/29/is-the-impact-factor-from-a-bygone-era/#comment-3751 Bev Acreman Thu, 02 Jul 2009 11:52:38 +0000 http://scholarlykitchen.sspnet.org/?p=4699#comment-3751 Bruce: this is something that the UKSG are investigating now - details at: http://www.uksg.org/usagefactors Bruce: this is something that the UKSG are investigating now – details at: http://www.uksg.org/usagefactors

]]> By: The Chutry Experiment » Tuesday Links http://scholarlykitchen.sspnet.org/2009/06/29/is-the-impact-factor-from-a-bygone-era/#comment-3731 The Chutry Experiment » Tuesday Links Tue, 30 Jun 2009 20:37:53 +0000 http://scholarlykitchen.sspnet.org/?p=4699#comment-3731 [...] The Scholarly Kitchen has an interesting discussion of the Impact Factor, a tool used to determine how much impact a sholarly book or journal has based on the number of citations in peer-reviewed journals.  The Schoarly Kitchen argues that such rubrics are now obsolete, given that citations can now appear anywhere, including tweets, blog posts, and Facebook status updates.  While these citations may not be equivalent with a mention in a peer-reviewed journal, theyoften do come from peers in the field.  This is something we’ve been talking about for a long time now at MediaCommons, but I think it is worth highlighting othes who are thinking about the ways in which digital media are enmeshed with questions of scholarly impact. [...] [...] The Scholarly Kitchen has an interesting discussion of the Impact Factor, a tool used to determine how much impact a sholarly book or journal has based on the number of citations in peer-reviewed journals.  The Schoarly Kitchen argues that such rubrics are now obsolete, given that citations can now appear anywhere, including tweets, blog posts, and Facebook status updates.  While these citations may not be equivalent with a mention in a peer-reviewed journal, theyoften do come from peers in the field.  This is something we’ve been talking about for a long time now at MediaCommons, but I think it is worth highlighting othes who are thinking about the ways in which digital media are enmeshed with questions of scholarly impact. [...]

]]>