Richard Smith: Peer reviewers—time for mass rebellion?

I’ve spent about six hours over the last two days reviewing two scientific papers, and the experience has made me wonder if it is time for peer reviewers to rise up in rebellion—rather as walkers made a mass trespass of Kinder Scout in 1932. As I write this, I remember one of Britain’s leading scientists saying to me: “I never peer review. Why would I waste my time reviewing crummy research when I can be doing my own research? What’s more, I’m funded to do research and am rewarded for it. Nobody either funds me to review or rewards me for doing it.”

Usually, I follow the message of the scientist and decline to peer review, but then I succumbed twice in 24 hours. In one case I was asked to review a paper that might be published alongside something I’ve written, and the other paper was about an issue I’m keen to advance. I was asked to review the papers within seven days and was chased after seven days. I did one after two weeks and another after 10 days, and as well as reading the papers and writing my reviews I had to grapple with the clunky software when submitting my reviews.

Although I took the reviews seriously and spent six hours doing them, I can’t feel that I added much if any value. Both papers were clearly written and well-evidenced. They could have been published immediately, and no harm would have been done. Instead, much time and bureaucracy has come before publication. (I accept that there are papers that would be better not published, but I suggest that the editors know which they are and that they are usually published anyway if the authors persist: you can get anything published if you keep going, as a series of nonsense papers have shown.)

There is an extra irony in me having peer reviewed these papers as I have long argued, based on evidence, that prepublication peer review is useless. There is almost no evidence of the effectiveness of peer review but substantial evidence that it is slow, expensive, poor at detecting errors, largely a lottery, prone to bias and abuse, unable to guard against fraud, and anti-innovatory in that it tends to reject truly original research. If it was a drug, the saying goes, it would never be approved.

Unfortunately demand for peer reviewers is growing. The number of journals is again increasing exponentially. When I was a publisher it was financially risky to launch a new journal: you immediately had the cost of producing, printing, and circulating the journal, and you had to market it. It would take three years to make a profit if you ever did. But now that journals are electronic only and the business model is the authors paying for publication you can make a profit almost immediately—you have the software to start a journal, and once you’ve published a few papers you are in profit.

Editors will tell you that it’s getting harder and harder to find reviewers, and I saw a Tweet a few days ago from a youngish scientist who had received eight requests to peer review in one week. There is also a perverse incentive for peer reviewers in that the better and faster you review the more you are asked to review. If you are the editor of a prestigious journal it isn’t so hard to find reviewers in that they may be flattered to be asked or may (probably wrongly) think that if they oblige the journal it will be more sympathetic when considering their next submission.

Reviewers are not paid to review, although when I started at The BMJ in 1979 we paid reviewers £50 for a review. Even then it was a token amount. I charge £1000 a day when I (rarely) consult (and I could probably charge £3500 if I had the chutzpah), so I have foregone something between £750 and £2625 by reviewing my two papers. I have made a contribution to the publishers, and I know that many journals and science publishers have a profit margin of 30%—far higher than the 3% of supermarkets or the 7% of the oil and gas drilling industry. Science publishers get their “oil” (scientific studies) for free and manage to avoid paying many of their “suppliers” (for example, peer reviewers).

The time has come for peer reviewers to rebel, but what is it we want? My preference would be that we refuse to review unless the review is of a paper that is already posted for all to see and that our reviews are also immediately posted for all to see. I can see, however, that this might be scary for young or new reviewers, so they should insist on a proper payment or some other real reward, not, for example, being listed along with a hundred others once a year in the journal, a certificate, or the offer of a discount on buying the publisher’s books or journals.

Currently, however, peer reviewers are powerless, often labouring anonymously at night and paid less than Victorian children sent up chimneys. Just like the Tolpuddle martyrs of 1834 we need to start a union, hoping that we are not like them transported before being pardoned. Who will take the lead?

Richard Smith was the editor of The BMJ until 2004.