You don't need to be signed in to read BMJ Blogs, but you can register here to receive updates about other BMJ products and services via our site.

Liz Wager: If comment is cheap why is peer review so expensive?

16 Apr, 09 | by julietwalker

Liz WagerAs you know (since you are reading this), I blog, albeit sporadically. I do not Tweet (yet) but I’m fascinated by the frenzy of twittering and the explosion of opportunities to launch one’s opinions into cyberspace. 

The BMJ offers Doc2Doc, which, along with similar sites, bubbles with medical comment and chat. Not being a Doc, I haven’t joined this particular conversation, but was interested to hear, at a BMJ Editors meeting, that Doc2Doc users were discussing abstracts from a meeting. Yet at a different meeting, I had heard editors from another journal bemoaning the fact that they have to send out 8 requests to get one review. That lead me to wonder why it is apparently so easy to get people (even busy people like doctors) to chat, but so hard to persuade them to peer review.

Some of the reasons are obvious. A tweet takes seconds, but a good review can take several hours. Tweets are limited to 140 characters, but constructive reviews are often several pages. But I wonder if there are other features of e-chat that editors could learn from? One is rapid response. Twittering feels like a real conversation because people reply. Yet peer review feels like talking to yourself. Even with open (ie signed) review, journals discourage authors and reviewers from getting in touch. Some journals let you see the other reviewers’ comments, but usually several weeks or months later. Reviewers sometimes see the authors’ response but this usually feels as if it were snarled through gritted teeth and any expressions of thanks to the reviewer usually seem formulaic and flattering. Wouldn’t it be nice if we could have a real dialogue? If reviewers had to meet authors face-to-face, or even across a video link, perhaps both sides would be more polite and more constructive.

Some journals encourage post-publication comment and annotation. But so far, readers don’t seem to be bothered and the quality of comments has been low. Bold experiments with ‘open’ (ie web-based) review (eg at the Medical Journal of Australia and, more recently, Nature) have flopped. This is disappointing, because such democratic critique sounds like a wonderful alternative or addition to conventional peer review. I’m convinced that one day, some clever editor will discover a neat way of harnessing the electronic twittering so that it contributes to the selection and refinement of articles. But it hasn’t happened yet.

Liz Wager is a freelance writer, trainer and publications consultant who works for a number of pharmaceutical companies, communication agencies, publishers and academic institutions. She is also the Secretary of COPE (the Committee On Publication Ethics) and a member of the BMJ’s Ethics Committee.

By submitting your comment you agree to adhere to these terms and conditions
  • http://www.pallimed.org Christian Sinclair, MD

    Medical journals could do a lot for opening the scientific discussion by highlighting when their articles are being discussed on other sites. Research Blogging (.org) is a great aggregator of insightful commentary but niche scientific and medical bloggers. Imagine if journal publishers supported that system. Part of your impact factor could be showing how much your articles are in the new, or on certain blogs.

  • http://www.pallimed.org Christian Sinclair, MD

    Of course, BMJ already does that (news and blog aggregating), so that is where I got my point from. But wouldn’t that be helpful to put on each articles web page too?

  • http://www.semeioticabiofisica.it Sergio Stagnaro

    Dear Liz, the argument, you illustrtates efficaciously, is really important, regarding scientific development. In a lot of Nature’s blogs it was already discussed in details, as you surely know. “If reviewers had to meet authors face-to-face, or even across a video link, perhaps both sides would be more polite and more constructive”. I agree with you completely, as I wrote elsewhere. In fact, as demonstrates real new progress – e.g., no local realm in biological systems, beside the local realm, at the base of Lory’s Experiment (http://blogs.nature.com/news/thegreatbeyond/2008/08/exhibit_a_religion_does_get_in.html#comments),
    reviewers may encounter sometimes a lot of difficulty reading such as argument, at least initially! How many epochal, revolutionary papers, which subsequently will play a central role in Science (in our case, Medicine) History, unfortunately are rejected too fast, due to erroneous misunderstanding: Jenner docet!
    If you like read something of this nature, visit kindly the Blog http://sciphu.com (sic.without www!).

  • http://www.fysioterapeuten.no Kjartan Vaarbakken

    In Norway today, where I’m editing science, the Universities and University Collages receive credit (points and money) for publishing. Different value for journals with different impact factors, of cause. Probably the practice is the same in other western countries.

    Anyway, as an editor I think it is about time that the scientists who peer-review and their institution get real credit for the effort. Not as much credit as for publishing, of cause, but something in the pocket and on their CV or record. This, I think, will really improve scientific editors efficiency. As for today, the most-publishing and -winning scientist is the one saying no, which is truly unjust system that waste the time for editors and authors. Therefore I call for a change in the crediting system, including peer-reviewing.

  • http://ww.plosone.org Peter Binfield

    Christian

    Interesting you make the following requests:

    >Medical journals could do a lot for opening the scientific
    >discussion by highlighting when their articles are being
    > discussed on other sites

    >But wouldn’t that be helpful to put on each articles web
    >page too?

    The PLoS journals just started doing exactly that, 2 weeks ago. See this blog post for details:

    http://everyone.plos.org/2009/03/31/newly-launched-features-on-our-online-platform/

    And this article for an example of it in action:

    http://www.plosone.org/article/related/info%3Adoi%2F10.1371%2Fjournal.pone.0001230

    Pete Binfield
    (Managing Editor of PLoS ONE)

  • http://www.sciencesurvivalblog.com Ad Lagendijk

    One of my PhD students recently blogged on our blog about post-publication comment and annotation (http://www.sciencesurvivalblog.com/getting-published/opendiscussion_584). He very much would like to see this after-publication happening. My problem (with a stack of papers by relativity-theory-deniers on my desk) is that it would only work if we could keep the crackpots out without falling back on old-fashioned peer-review.

  • Liz Wager

    Kjartan makes a good point. Peer review (if done well) is hard work, and it ought to be rewarded, but, ironically, if it were, that would make the process even more costly! Some journals now issue certificates to acknowledge the work of peer reviewers but I agree that academic recognition is needed.

  • http://bmj.com Trish Groves

    Here’s what the BMJ offers reviewers (see http://resources.bmj.com/bmj/reviewers):

    “To thank reviewers we offer a year’s online subscription to bmj.com and, if a reviewer already has access to bmj.com, we encourage them to pass this on to a colleague. We also provide reviewers with a Journal Activity Certificate for the BMJ. This keeps track of the work reviewers do for the BMJ and enables them to print off a certificate. We hope that this certificate may be helpful for reviewers who need to record reviewing as part of their continuing professional development activity.
    Peer review is an important academic role that deserves recognition. To that end, we list the names of BMJ reviewers each year on bmj.com.”

  • http://www.sciencesurvivalblog.com Ad Lagendijk

    As an long-time editor (11 years) of an Elsevier journal I have some experience with the difficulties in getting good referee reports. As I explained in my post http://www.sciencesurvivalblog.com/miscellaneous/should-we-pay-referees_71 a good solution would indeed be paying the referees. Elsevier always came up with new answers why they did not want to do this.

  • http://www.fysioterapeuten.no Kjartan Vaarbakken

    I thank Liz for the support and Trish for useful suggestions.

    I’m sure Trish knows about it, but I would like to mention a study by Tite and Schroter (2007), working at BMJ Editorial Office. The question they posed was “Why do peer-reviewers decline to review?”

    They concluded: Reviewers are more likely to accept to review a manuscript when it is relevant to their area of interest. Lack of time is the principal factor in the decision to decline. Reviewing should be formally recognized by academic institutions and journals should acknowledge reviewers’ work.

    I find their results even more interesting. Responders were 62 % (551/890) of the reviewers in five BMJ-group journals. Factors rated most highly in importance for the decision to accept to review a paper were; contribution of the paper to subject area, relevance of topic to own work, and opportunity to learn something new. Most respondents agreed that financial incentives would not be effective when time constraints are prohibitive.

    I do hope the Journal Activity Certificate promote reviewers career opportunities and the BMJ’s positive response rate from reviewers. The certificate seems like a good idea.

You can follow any responses to this entry through the RSS 2.0 feed.
BMJ blogs homepage

The BMJ

Helping doctors make better decisions. Visit site



Creative Comms logo

Latest from The BMJ

Latest from The BMJ

Latest from BMJ podcasts

Latest from BMJ podcasts

Blogs linking here

Blogs linking here