The Franciscan philosopher Roger Bacon (c1214-1294), who some regard as the father of modern science, argued in his great text Opus Majus that there were four sources of ignorance:
- Frail and unsuited authority
- The influence of custom
- The opinion of the unlearned crowd
- The concealment of our ignorance in a display of apparent wisdom
I must confess right away that I didn’t read this in the Latin of Opus Majus but in the English of Bertrand Russell’s History of Western Philosophy. It’s important that I confess this as I’m taking Russell as an authority (or at least a reliable messenger), which Bacon criticised, and because I haven’t and couldn’t read the original Latin—and Bacon was a strong proponent of learning languages and studying the original texts.
When I read Bacon’s four causes of ignorance, I thought immediately of peer review. It’s clear to me that Bacon would have dismissed peer review.
Firstly, peer review depends entirely on “frail and unsuited authority.” It has no evidence base in the scientific sense of experimental studies demonstrating its effectiveness. It’s important because authorities say it is, and those authorities are, I suggest, “unsuited.” They are unsuited because they may be distinguished physicists or biochemists but they have not studied and experimented with peer review.
Secondly, peer review depends wholly on custom. It may not be as old as people think, but it is “the way we do things around here” in deciding on which grant proposals will be funded and which papers published where. And there are now huge vested interests: profits made, jobs provided, and reputations built. Bacon thought that concealment of our ignorance in a display of apparent wisdom was the main source of ignorance, and T S Eliot would agree: “In order to arrive at what you do not know/You must go by a way which is the way of ignorance.” But custom (the way we do things here)—with its failure to recognise, and to even celebrate, ignorance—runs a close second.
Thirdly, the “unlearned crowd”—editors of journals, authors of Scholarly Kitchen, publishers of profitable journals, and sadly most scientists—insist that peer review must be central to science.
Fourthly, scientific authorities conceal their ignorance in a display of apparent wisdom. I can’t claim to know everything about peer review, but I have been following evidence on peer review for nearly 40 years. That evidence, as I’ve written many times, fails to show effectiveness (and I accept that absence of evidence of effectiveness is not the same as evidence of absence of effectiveness) but does show that peer review is slow, expensive, something of a lottery, inefficient, wasteful of scientific resource, poor at detecting error and fraud, prone to bias, and anti-innovatory. Yet when I present this evidence to the learned—as I have done at the Royal Society, UNESCO, and other theatres of supposed wisdom—it is dismissed with anecdote: “I’ve had many papers improved by peer review.”
I’m sure that if Roger Bacon were to return from the grave he would condemn peer review outright.
Richard Smith was the editor of The BMJ until 2004.
Competing interests: None declared.