Liz Wager: Are journal editors like used car salesmen?

Liz WagerYesterday, I gave evidence to a UK parliamentary inquiry into peer review (as did Fiona Godlee). (The session can be viewed here)

Before the session I tried to think of an analogy for peer review that I could use to explain its usefulness, but also its variety and imperfections, to the MPs. Inspiration often comes from unlikely sources – in this case an episode of the 1980s TV sitcom “Minder” which is based around the dodgy dealings of secondhand car salesman Arthur Daley (about as unlikely a role model for a journal editor as you can find, but I hope Fiona and her fellow editors will forgive the comparison).

So, what has peer review got to do with selling used motors? In the UK, all cars over three years old have to undergo an annual inspection called the MOT test (named after the now nonexistent Ministry of Transport). Here was my analogy. The test is designed to reduce breakdowns and keep traffic flowing. It’s not perfect and it doesn’t guarantee that every tested car will run smoothly for the next 52 weeks but it probably keeps the worst wrecks off the roads and often identifies problems that can be fixed (just like peer review). It’s considered reasonable and is therefore acceptable to most law abiding motorists. Of course, more breakdowns could be prevented if cars had to be tested once a month instead of once a year, but that would be disproportionate and would mean cars were off the road too often while they were being tested. (Similarly, peer review could be made tougher by insisting that journals review or re-analyse the raw data, but few people would consider this acceptable and it would cause substantial delays.)

Another reason why I was pleased with the MOT analogy is that, just like peer review, the test doesn’t always spot deliberate fraud such as a mileometer that’s been “clocked” (i.e. turned back to show fewer miles than the car has actually travelled). If you were buying a used car you would probably want to know more than the fact that it had passed its MOT. You might go to a reputable dealer or get it checked by a friendly mechanic. Users of articles should probably do the same and I guess if the journal editors are the dealers, then statisticians and methodologists are the friendly mechanics who check your potential new purchase and warn you of its imperfections.

And, lastly, you need an MOT whether you drive a Rolls Royce or a grubby white van – the MOT doesn’t tell you which is which. Similarly, peer review is used by a huge range of journals, some of which are highly selective and some of which will publish almost anything so long as it is “roadworthy” (i.e. valid).

So, in one of Arthur’s own splendid malapropisms I’ve now got a nice “allergy” for peer review. But, despite my own South London roots, I’ll resist the temptation of lapsing into rhyming slang for my next blog but challenge readers to suggest a nice Cockney name for peer review.

Liz Wager PhD is a freelance medical writer, editor, and trainer. She is the current chair of the Committee on Publication Ethics (COPE).

  • Another analogy would be related to drugs. Some editors are drug pushers, others are apothecaries. Publishers are like big pharma or cocaine cartels. Authors are chemists creating drugs – some useful, some harmful, some original, some copycat. Peer reviewers test the products and offer opinions on how good the drug is.

  • My ancestors came to America from Wales in the mid 1600s, so any British/Welsh/Cockney is certainly about as diluted as free drinks at an open bar…. that being stated… almost anything rhymes with Peer Review…. so how about “Gone to the Loo”… which is where much of the reading/reviewing may be actually done… 😉  great blog and thanks for posting….

  • Mark Thornton

    Re: rhyming slang . . “Rabbit Stew = Peer review” ??? As in “that's a nice bit of Rabbit Stew”.

  • Liz Wager

    Thanks for the suggestions — funnily enough I had stew on the brain and the first one I thought of was 'Irish stew', but I like the lavatorial suggestion as well!

  • Richard Smith

    Arguing by analogy can be helpful an fun but ultimately misleads, as it
    does here, I fear. My predecessor as editor of the BMJ, Stephen Lock, used to
    say that being an editor was “taking in other people's washing.” He also said
    that most of what is published has “the impact of a doughnut in the North

  • Douglas Carnall

    The MoT test was introduced in 1960 not “to prevent breakdowns and keep traffic flowing” but because so many vehicles on the road had defective brakes, lights and steering and were a hazard to others. Peer review doesn't take the crocks off the road, it merely displaces them to less travelled by-ways where they can be safely ignored.
    Publication in peer reviewed journals is much more about mediating competition for scarce resources, generating hierarchy and establishing the dominant discourse than it is about communication or verification of knowledge per se.A better analogy might be a department store on a busy city corner which has plentiful footfall but limited shelf space. Naturally the manager wishes to stock the shop with the classiest goods available. Fortunately, the customers queue up to stock the shelves for free, then pay at the cashdesk for the privilege of taking home a fraction of what's on offer. Great business!”Rabbit stew” for “peer review” is rather fun, though “rabbit and pork” for talk is already well established. Perhaps “Pay-per-view” would be more apposite?