You don't need to be signed in to read BMJ Group Blogs, but you can register here to receive updates about other BMJ Group products and services via our Group site.

Richard Smith: Enter the “liquid journal”

5 Aug, 10 | by BMJ Group

Richard SmithIt may be what epidemiologists call “ascertainment bias” (seeing what you want to see), but I detect the beginning of the end of prepublication peer review. The latest death knell is the appearance of a “liquid journal” where scientists can post papers without peer review and papers in evolution, data sets, pieces of computer code, or blogs. The new journal is a research project funded by the European Union and supported by the French National Centre for Scientific Research, Springer Science (a major commercial publisher), and others.

The new journal (and maybe “journal” is not the best word for such a new thing) is the idea of Fabio Casati, professor of computer science at the University of Trento and the holder of 20 patents. Casati thinks that scientists are spending too much time writing papers, many of them describing tiny incremental developments, and not enough time doing science. The incentives are all wrong: scientists are rewarded for publishing in major journals and so spend too much time writing papers. “The more papers you produce, the more brownie points you get,” says Casati. “So most of your time is spent writing papers instead of thinking or doing science.”

Casati wants to bring the doing and the disseminating of science closer together. We saw this at work in the Human Genome Project where scientists were constantly exchanging data through the web and where the final papers in Science and Nature were like the topping off of a cathedral rather than the building of the cathedral.

The first liquid journal appropriately is on peer review, and Casati and his colleagues have done research which convinces them that prepublication peer review is worthless. “We’ve studied this and found that peer review doesn’t work, in the sense that there seems to be very little correlation between the judgement of peer reviewers and the fate of a paper after publication,” says Casati. “Many papers get very high marks from their peer reviewers but have little effect on the field. And on the other hand, many papers get average ratings but have a big impact.”

Like me Casati believes that the real peer review comes after publication. The world will decide in the market of ideas which papers matter most.  “If you and I include this paper in our journals, we are giving it value,” says Casati. “When this is done by hundreds of people like us, we’re using the selection power of the entire community to value the contribution. Interesting papers will rise above the noise.” This is “we think” rather than what a few arbitrarily selected reviewers think.

As I write this I think of a sad—indeed, ridiculous—story I’ve just heard of the failures of peer review. It concerns a paper that I’ve known about for two years and think very important that still hasn’t been published. It’s a paper written by a fellow of the Royal Society with others, and none of the perhaps 15 reviewers of the three journals who have rejected the paper have seen a major flaw. It is, however, a paper that is “disturbing” in that it suggests that what a lot of researchers (read “peer reviewers”) are doing is a waste of time. It seems to me that the peer review process has been a terrible waste of time, effort, and talent. Much better to have posted the paper on the web and let the world decide its importance or lack of it and for the reviewers to have got on with researching.

The chances are, however, that Casati’s journal will not be the development that causes the whole edifice of peer review and traditional journals to come crashing down—there are too many opposing forces.

Firstly, scientists are surprisingly conservative. I never quite understand why they should be so when their whole way of working is questioning, experimenting, criticising, and doubting—but they are.

Secondly, rewards still come from publishing in the major journals, all of them making substantial profits from value added by others—the scientists themselves. Universities have been talking about “uncoupling” evaluation of performance and publication for 20 years—but it hasn’t happened.

Thirdly, there are too many vested interests to overcome. Publishers, both commercial and scientific societies, and their employees have their empires, profits, and jobs invested in peer review and the traditional journals.

Yet despite these opposing forces the new ways opened up by the internet will eventually bring down prepublication peer review and transform traditional journals just as they have brought down oppressive regimens. I urge you all to access the new journal at http://project.liquidpub.org/ and email Fabio Casati at Fabio.Casati@gmail.com offering support. I’ve already done so.

Competing interests: RS is on the board (unpaid) of the open access publisher, Public Library of Science, and has written a book, the Trouble with Medical Journals in which he criticises peer review and traditional journals. If you were to buy a copy he’d made £2, but he’d be just as happy if you borrowed it from the library as he doesn’t need the money.

By submitting your comment you agree to adhere to these terms and conditions
  • Michael Millenson

    The idea of “crowdsourcing” academic review is attractive, but the reality may show quite a few drawbacks.

    Will anyone be allowed to comment (or, better, any entity that has a “bot” to post comments)? Even before the Web became popular, early medical listservs on then-obscure topics such as informatics collapsed under the weight of extraneous comments.

    Who will take the time to comment? Finding qualified peer reviewers is already difficult. When something becomes everyone's responsibility, it can quickly become no one's responsibility — except for those with a vested economic interest or ideological point of view.

    The core idea of a scientific community providing intellectual self-regulation in near time is seductive. All utopias are.

  • Thkellner

    Very interesting subject. I just went through the same experience. We published a study design, it turned out as being one of the most read study design publications in an open access trial journal. We saw that as a signal for broad interest in the topic.

    Based on this experience we have chosen a well known journal for final publication. We got good comments from peer reviewers, finnally it was rejected by the editor as being of minor relevance. I can just confirm the above – big effort in time and ressources for the publication, we spent almost more time on the publication then on the research subject itself.

    However, there is still a need for some sort of quality control in the editing process that needs to be assured. It can not solely be based on the readers to judge on the quality. A good opportunity for something new.

  • Peter Arnold

    Richard, I agree especially with the idea of placing all research data on the Internet, in searchable spreadsheet form, letting others analyse the material in ways which fit in with their areas of interest. The original researchers are usually looking for data to confirm or reject a specific hypothesis in their own area; and may easily overlook correlations in other areas – of interest to other researchers. Making the raw data available to all would also facilitate meta-analyes.
    Peter Arnold, Sydney, Australia

  • http://www.badscience.net bengoldacre

    v interesting website, but not very attractive/negotiable, it took me a while to find where the papers are, the pages loaded slowly for me (might be a blip), and it's a shame that papers can't be tagged / organised with keywords (maybe they can and i missed it).

    overall i guess maybe we need both trad journals and spaces where ppl can say things without peer review, like Bruce Charlton argued:

    http://www.bmj.com/cgi/content/extract/335/7617…

    although the trouble is these journals (like BC's Medical Hypotheses) can end up being magnets for silliness:

    http://www.badscience.net/2009/09/medical-hypot…
    http://www.timeshighereducation.co.uk/story.asp…
    http://www.badscience.net/2008/10/more-crap-jou…
    http://www.badscience.net/2007/08/observations-…

    (sorry for links to my own site)

  • Richard Smith

    You can read more on the liquid journal here: http://www.the-scientist.com/blog/display/57613/

  • Talyam

    Dear Richard,
    Thanks for sharing. While open access cannot be the panacea to every academic evil, the very act of offering an alternative to existing practices helps highlight how we consider peer reviewed publications as the be all and end all of science. Such publications are the main channel for transferring ideas, as well as the means by which importance is established, tenure is granted and merit is measured. Anyone who ever had anything to do with the peer review process knows it is often purely objective, yet sometimes anything but. A favorable editor may suggest resubmission if you change every premise in the paper, while a begrudging reviewer may deem your paper worthless. Likewise, bland reviews can translate to acceptance or rejection depending on what seems very much like a whim, masquerading as good science and measured reason.
    As the cast of Hair sings: “let the sun shine in”. Let us put our ideas out there for everyone to see, evaluate, appreciate or denounce, and may the better ideas win.

  • Liz Wager

    This is interesting but I'm not convinced its the harbinger of a massive change in medical publishing. Like Ben Goldacre, I couldn't easily find the articles and the reading experience was grim. And I wonder if clinicians really want to read liquid versions of clinical trials — wouldn't they prefer to wait for a more polished version? As you know, I'm no fan of conventional peer review and in favour of new models of publishing (1). I also think we need to be careful that vested interests don't slow progress (2). But I'm not convinced that this is the way to go.

    Refs
    1 Wager (2006) Publishing clinical trial results: the future beckons. PLoS Clinical Trials 1(6):e34
    2 Wager & Abbasi (2009) Medical editors and trial reporting: a betrayal of patient care. JRSM 101:4-5

  • susanne stevens mccabe

    The suggestion of borrowing books from the library rather than buying is useful, especially those books which are unlikely to be read again by an individual. Libraries are experiencing cuts though and will not always be able to add new books on request or at a time they are needed. Maybe there are schemes already but it would be good if some kind of free book service could be set up. Lists of unwanted books could be submitted and offered on a first come first served basis. The well off could easily,altruistically afford to pay for postage and package so that knowledge could be shared more widely. Authors might donate some of their books for free.

  • http://profiles.yahoo.com/u/UANBMT6DG3LRTUPXAUKMTG2TXI Proxies Century

    If you are looking for paper dispenser for home or office, the options are many. You can find a wide variety of kitchen or bathroom. With manual and automatic self-feeding, all available in different price ranges, it is really down, depends on personal preference. Visit http://www.papertoweldispenser.biz

  • dpayne

    I will be out of the office starting 26/08/2010 and will not return until
    13/09/2010.

    For urgent bmj.com queries, please contact Birte Twisselmann or Juliet
    Walker, and for doc2doc.bmj.com, please contact Sabreena Malik

    _______________________________________________________________________
    The BMJ Group is one of the world's most trusted providers of medical information for doctors, researchers, health care workers and patients group.bmj.com. This email and any attachments are confidential. If you have received this email in error, please delete it and kindly notify us. If the email contains personal views then the BMJ Group accepts no responsibility for these statements. The recipient should check this email and attachments for viruses because the BMJ Group accepts no liability for any damage caused by viruses. Emails sent or received by the BMJ Group may be monitored for size, traffic, distribution and content. BMJ Publishing Group Limited trading as BMJ Group. A private limited company, registered in England and Wales under registration number 03102371. Registered office: BMA House, Tavistock Square, London WC1H 9JR, UK.
    _______________________________________________________________________

You can follow any responses to this entry through the RSS 2.0 feed.
BMJ blogs homepage

BMJ.com

Helping doctors make better decisions. Visit site



Creative Comms logo

Latest from BMJ.com

Latest from BMJ.com

Latest from BMJ.com podcasts

Latest from BMJ.com podcasts

Blogs linking here

Blogs linking here