{"id":1280,"date":"2013-10-15T16:47:10","date_gmt":"2013-10-15T15:47:10","guid":{"rendered":"https:\/\/blogs.bmj.com\/injury-prevention\/?p=1280"},"modified":"2013-10-15T16:47:10","modified_gmt":"2013-10-15T15:47:10","slug":"impact-factor-revisited","status":"publish","type":"post","link":"https:\/\/blogs.bmj.com\/injury-prevention\/2013\/10\/15\/impact-factor-revisited\/","title":{"rendered":"Impact factor revisited"},"content":{"rendered":"

I have made my feelings about the impact factor well known but still, far too many cling to it as if it were some sort of holy grail. Now it seems I was both wrong and right: it appears to be both deeply flawed but better than any of the alternatives. Readers should weigh in if they disagree. \u00a0The following is from a recent publication in PLOS via Retraction Watch. The paper examined IF alongside other methods of peer review:<\/p>\n

Retraction Watch<\/p>\n

\"plos<\/a>We\u2019ve sometimes said,\u00a0paraphrasing Winston Churchill<\/a>, that pre-publication peer review is the worst way to vet science, except for all the other ways that have been tried from time to time.<\/p>\n

The authors of a new paper in\u00a0PLOS Biology<\/em>,\u00a0Adam Eyre-Walker<\/a>\u00a0and\u00a0Nina Stoletzki<\/a>, compared\u00a0three of those other ways<\/a>\u00a0to judge more than 6,500 papers published in 2005:<\/p>\n

subjective post-publication peer review, the number of citations gained by a paper, and the impact factor of the journal in which the article was published<\/p><\/blockquote>\n

Their findings?<\/p>\n

We conclude that the three measures of scientific merit considered here are poor; in particular subjective assessments are an error-prone, biased, and expensive method by which to assess merit. We argue that the impact factor may be the most satisfactory of the methods we have considered, since it is a form of pre-publication review. However, we emphasise that it is likely to be a very error-prone measure of merit that is qualitative, not quantitative.<\/p><\/blockquote>\n

\n

(Disclosure: Ivan worked at Thomson Reuters, whose Thomson Scientific division owns the impact factor, from 2009 until the middle of this year, but was at Reuters Health, a completely separate unit of the company.)<\/p>\n

Or, put another way, as Eyre-Walker\u00a0told<\/a>\u00a0The Australian<\/em>:<\/p>\n

Scientists are probably the best judges of science, but they are pretty bad at it.<\/p><\/blockquote>\n

In an\u00a0accompanying editorial<\/a>,\u00a0Jonathan Eisen<\/a>,\u00a0Catriona MacCallum<\/a>, and\u00a0Cameron Neylon<\/a>\u00a0call the paper \u201cimportant\u201d and acknowledged that the authors found that impact factor \u201cis probably the least-bad metric amongst the small set that they analyse,\u201d but note some limitations:<\/p>\n<\/div>\n

The subjective assessment of research by experts has always been considered a gold standard\u2014an approach championed by researchers and funders alike\u00a0[3]<\/a>\u2013[5]<\/a>, despite its problems\u00a0[6]<\/a>. Yet a key conclusion of the study is that the scores of two assessors of the same paper are only very weakly correlated (Box 1<\/a>). As Eyre-Walker and Stoletzki rightly conclude, their analysis now raises serious questions about this process and, for example, the ~\u00a360 million investment by the UK Government into the UK Research Assessment Exercise (estimated for 2008), where the work of scientists and universities are largely judged by a panel of experts and funding allocated accordingly. Although we agree with this core conclusion and applaud the paper, we take issue with their assumption of \u201cmerit\u201d and their subsequent argument that the IF (or any other journal metric) is the best surrogate we currently have.<\/p><\/blockquote>\n

We have, as Retraction Watch readers may recall,\u00a0extolled the virtues of post-publication peer review<\/a>\u00a0before.<\/p>\n","protected":false},"excerpt":{"rendered":"

I have made my feelings about the impact factor well known but still, far too many cling to it as if it were some sort of holy grail. Now it seems I was both wrong and right: it appears to be both deeply flawed but better than any of the alternatives. Readers should weigh in […]<\/p>\n

Read More…<\/a><\/p>\n","protected":false},"author":118,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2574],"tags":[],"class_list":["post-1280","post","type-post","status-publish","format-standard","hentry","category-journal-writing"],"yoast_head":"\nImpact factor revisited - Injury Prevention: Editor's Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/blogs.bmj.com\/injury-prevention\/2013\/10\/15\/impact-factor-revisited\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Impact factor revisited - Injury Prevention: Editor's Blog\" \/>\n<meta property=\"og:description\" content=\"I have made my feelings about the impact factor well known but still, far too many cling to it as if it were some sort of holy grail. Now it seems I was both wrong and right: it appears to be both deeply flawed but better than any of the alternatives. Readers should weigh in [...]Read More...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/blogs.bmj.com\/injury-prevention\/2013\/10\/15\/impact-factor-revisited\/\" \/>\n<meta property=\"og:site_name\" content=\"Injury Prevention: Editor's Blog\" \/>\n<meta property=\"article:published_time\" content=\"2013-10-15T15:47:10+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/retractionwatch.files.wordpress.com\/2013\/10\/plos-biology.png?w=700\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/injury-prevention\\\/2013\\\/10\\\/15\\\/impact-factor-revisited\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/injury-prevention\\\/2013\\\/10\\\/15\\\/impact-factor-revisited\\\/\"},\"author\":{\"name\":\"\",\"@id\":\"\"},\"headline\":\"Impact factor revisited\",\"datePublished\":\"2013-10-15T15:47:10+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/injury-prevention\\\/2013\\\/10\\\/15\\\/impact-factor-revisited\\\/\"},\"wordCount\":497,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/injury-prevention\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/injury-prevention\\\/2013\\\/10\\\/15\\\/impact-factor-revisited\\\/#primaryimage\"},\"thumbnailUrl\":\"http:\\\/\\\/retractionwatch.files.wordpress.com\\\/2013\\\/10\\\/plos-biology.png?w=700\",\"articleSection\":[\"Writing for Publication\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/blogs.bmj.com\\\/injury-prevention\\\/2013\\\/10\\\/15\\\/impact-factor-revisited\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/injury-prevention\\\/2013\\\/10\\\/15\\\/impact-factor-revisited\\\/\",\"url\":\"https:\\\/\\\/blogs.bmj.com\\\/injury-prevention\\\/2013\\\/10\\\/15\\\/impact-factor-revisited\\\/\",\"name\":\"Impact factor revisited - Injury Prevention: Editor's Blog\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/injury-prevention\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/injury-prevention\\\/2013\\\/10\\\/15\\\/impact-factor-revisited\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/injury-prevention\\\/2013\\\/10\\\/15\\\/impact-factor-revisited\\\/#primaryimage\"},\"thumbnailUrl\":\"http:\\\/\\\/retractionwatch.files.wordpress.com\\\/2013\\\/10\\\/plos-biology.png?w=700\",\"datePublished\":\"2013-10-15T15:47:10+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/injury-prevention\\\/2013\\\/10\\\/15\\\/impact-factor-revisited\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/blogs.bmj.com\\\/injury-prevention\\\/2013\\\/10\\\/15\\\/impact-factor-revisited\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/injury-prevention\\\/2013\\\/10\\\/15\\\/impact-factor-revisited\\\/#primaryimage\",\"url\":\"http:\\\/\\\/retractionwatch.files.wordpress.com\\\/2013\\\/10\\\/plos-biology.png?w=700\",\"contentUrl\":\"http:\\\/\\\/retractionwatch.files.wordpress.com\\\/2013\\\/10\\\/plos-biology.png?w=700\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/injury-prevention\\\/2013\\\/10\\\/15\\\/impact-factor-revisited\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/blogs.bmj.com\\\/injury-prevention\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Impact factor revisited\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/injury-prevention\\\/#website\",\"url\":\"https:\\\/\\\/blogs.bmj.com\\\/injury-prevention\\\/\",\"name\":\"Injury Prevention: Editor's Blog\",\"description\":\"The official blog for the journal Injury Prevention.\",\"publisher\":{\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/injury-prevention\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/blogs.bmj.com\\\/injury-prevention\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/injury-prevention\\\/#organization\",\"name\":\"Injury Prevention: Editor's Blog\",\"url\":\"https:\\\/\\\/blogs.bmj.com\\\/injury-prevention\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/injury-prevention\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/blogs.bmj.com\\\/injury-prevention\\\/files\\\/2017\\\/10\\\/blog-logo-injury-prevention.png\",\"contentUrl\":\"https:\\\/\\\/blogs.bmj.com\\\/injury-prevention\\\/files\\\/2017\\\/10\\\/blog-logo-injury-prevention.png\",\"width\":270,\"height\":34,\"caption\":\"Injury Prevention: Editor's Blog\"},\"image\":{\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/injury-prevention\\\/#\\\/schema\\\/logo\\\/image\\\/\"}},{\"@type\":\"Person\",\"@id\":\"\",\"url\":\"https:\\\/\\\/blogs.bmj.com\\\/injury-prevention\\\/author\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Impact factor revisited - Injury Prevention: Editor's Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/blogs.bmj.com\/injury-prevention\/2013\/10\/15\/impact-factor-revisited\/","og_locale":"en_US","og_type":"article","og_title":"Impact factor revisited - Injury Prevention: Editor's Blog","og_description":"I have made my feelings about the impact factor well known but still, far too many cling to it as if it were some sort of holy grail. Now it seems I was both wrong and right: it appears to be both deeply flawed but better than any of the alternatives. Readers should weigh in [...]Read More...","og_url":"https:\/\/blogs.bmj.com\/injury-prevention\/2013\/10\/15\/impact-factor-revisited\/","og_site_name":"Injury Prevention: Editor's Blog","article_published_time":"2013-10-15T15:47:10+00:00","og_image":[{"url":"http:\/\/retractionwatch.files.wordpress.com\/2013\/10\/plos-biology.png?w=700","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_misc":{"Written by":"","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/blogs.bmj.com\/injury-prevention\/2013\/10\/15\/impact-factor-revisited\/#article","isPartOf":{"@id":"https:\/\/blogs.bmj.com\/injury-prevention\/2013\/10\/15\/impact-factor-revisited\/"},"author":{"name":"","@id":""},"headline":"Impact factor revisited","datePublished":"2013-10-15T15:47:10+00:00","mainEntityOfPage":{"@id":"https:\/\/blogs.bmj.com\/injury-prevention\/2013\/10\/15\/impact-factor-revisited\/"},"wordCount":497,"commentCount":0,"publisher":{"@id":"https:\/\/blogs.bmj.com\/injury-prevention\/#organization"},"image":{"@id":"https:\/\/blogs.bmj.com\/injury-prevention\/2013\/10\/15\/impact-factor-revisited\/#primaryimage"},"thumbnailUrl":"http:\/\/retractionwatch.files.wordpress.com\/2013\/10\/plos-biology.png?w=700","articleSection":["Writing for Publication"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/blogs.bmj.com\/injury-prevention\/2013\/10\/15\/impact-factor-revisited\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/blogs.bmj.com\/injury-prevention\/2013\/10\/15\/impact-factor-revisited\/","url":"https:\/\/blogs.bmj.com\/injury-prevention\/2013\/10\/15\/impact-factor-revisited\/","name":"Impact factor revisited - Injury Prevention: Editor's Blog","isPartOf":{"@id":"https:\/\/blogs.bmj.com\/injury-prevention\/#website"},"primaryImageOfPage":{"@id":"https:\/\/blogs.bmj.com\/injury-prevention\/2013\/10\/15\/impact-factor-revisited\/#primaryimage"},"image":{"@id":"https:\/\/blogs.bmj.com\/injury-prevention\/2013\/10\/15\/impact-factor-revisited\/#primaryimage"},"thumbnailUrl":"http:\/\/retractionwatch.files.wordpress.com\/2013\/10\/plos-biology.png?w=700","datePublished":"2013-10-15T15:47:10+00:00","breadcrumb":{"@id":"https:\/\/blogs.bmj.com\/injury-prevention\/2013\/10\/15\/impact-factor-revisited\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/blogs.bmj.com\/injury-prevention\/2013\/10\/15\/impact-factor-revisited\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/blogs.bmj.com\/injury-prevention\/2013\/10\/15\/impact-factor-revisited\/#primaryimage","url":"http:\/\/retractionwatch.files.wordpress.com\/2013\/10\/plos-biology.png?w=700","contentUrl":"http:\/\/retractionwatch.files.wordpress.com\/2013\/10\/plos-biology.png?w=700"},{"@type":"BreadcrumbList","@id":"https:\/\/blogs.bmj.com\/injury-prevention\/2013\/10\/15\/impact-factor-revisited\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/blogs.bmj.com\/injury-prevention\/"},{"@type":"ListItem","position":2,"name":"Impact factor revisited"}]},{"@type":"WebSite","@id":"https:\/\/blogs.bmj.com\/injury-prevention\/#website","url":"https:\/\/blogs.bmj.com\/injury-prevention\/","name":"Injury Prevention: Editor's Blog","description":"The official blog for the journal Injury Prevention.","publisher":{"@id":"https:\/\/blogs.bmj.com\/injury-prevention\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/blogs.bmj.com\/injury-prevention\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/blogs.bmj.com\/injury-prevention\/#organization","name":"Injury Prevention: Editor's Blog","url":"https:\/\/blogs.bmj.com\/injury-prevention\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/blogs.bmj.com\/injury-prevention\/#\/schema\/logo\/image\/","url":"https:\/\/blogs.bmj.com\/injury-prevention\/files\/2017\/10\/blog-logo-injury-prevention.png","contentUrl":"https:\/\/blogs.bmj.com\/injury-prevention\/files\/2017\/10\/blog-logo-injury-prevention.png","width":270,"height":34,"caption":"Injury Prevention: Editor's Blog"},"image":{"@id":"https:\/\/blogs.bmj.com\/injury-prevention\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"","url":"https:\/\/blogs.bmj.com\/injury-prevention\/author\/"}]}},"_links":{"self":[{"href":"https:\/\/blogs.bmj.com\/injury-prevention\/wp-json\/wp\/v2\/posts\/1280","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.bmj.com\/injury-prevention\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.bmj.com\/injury-prevention\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.bmj.com\/injury-prevention\/wp-json\/wp\/v2\/users\/118"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.bmj.com\/injury-prevention\/wp-json\/wp\/v2\/comments?post=1280"}],"version-history":[{"count":0,"href":"https:\/\/blogs.bmj.com\/injury-prevention\/wp-json\/wp\/v2\/posts\/1280\/revisions"}],"wp:attachment":[{"href":"https:\/\/blogs.bmj.com\/injury-prevention\/wp-json\/wp\/v2\/media?parent=1280"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.bmj.com\/injury-prevention\/wp-json\/wp\/v2\/categories?post=1280"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.bmj.com\/injury-prevention\/wp-json\/wp\/v2\/tags?post=1280"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}