Here are three tales of retractions and a puzzle.
A tale of apparent benefit
In a tweet on 16 April, the prolific Professor Mandeep R Mehra, whose 2020 PubMed record already runs to 44 papers, announced a forthcoming paper. The preprint reported the results of an “observational propensity-matched case-controlled study” in 1408 patients with covid-19; 704 received a single dose of ivermectin 15 micrograms/kg and 704 did not. The in-hospital mortality rates were 1.4% versus 8.5% respectively (hazard ratio = 0.2; 95% CI = 0.11–0.37). The patients were said to have come from 169 hospitals in three continents, but a table in the preprint identified them as having come from North America (n=923), South America (69), Europe (227), Africa (66), Asia (121), and Australia (2).
Many comments on Reddit were enthusiastic. Only a few were skeptical. None questioned the provenance of the data. The preprint was later withdrawn as “not ready for peer review”.
A second tale of apparent benefit
On 1 May the New England Journal of Medicine published another paper by Mehra and four colleagues (Figure 1). They described an apparent beneficial effect of ACE inhibitors in covid-19 but no effect of angiotensin receptor blockers (ARBs). Their registry study was in 8910 patients in 169 hospitals in Asia, Europe, and North America, whose data had been gathered in an observational database called the Surgical Outcomes Collaborative (Surgisphere), described as “an international registry”, whose director is co-author Sapan Desai.
However, following an expression of concern on 2 June, all five authors wrote a letter to the journal: “Because all the authors were not granted access to the raw data and the raw data could not be made available to a third-party auditor, we are unable to validate the primary data sources underlying our article.” They therefore asked that the article be retracted.
Figure 1. Source https://pubmed.ncbi.nlm.nih.gov/32356626/
A tale of apparent harm
On 22 May The Lancet published another paper from Mehra and three colleagues (Figure 2) describing a registry study of data from 671 hospitals worldwide in more than 96 000 patients with covid-19, of whom 16% been treated with a 4-aminoquinoline, chloroquine or hydroxychloroquine. The reported mortality was 7530/81144 (9%) in patients who received neither drug and 3168/14888 (21%) in those who received a 4-aminoquinoline; those who received azithromycin or clarithromycin in addition were at higher risk of dying (2318/10004, 23%) than those who received a 4-aminoquinoline alone (850/4884, 17%).
Figure 2. Source https://pubmed.ncbi.nlm.nih.gov/32450107
Suddenly the floodgates opened. Australian hospitals reported that they had not given permission for any data to be released, and hospitals elsewhere reported likewise. Unusual features of the paper were highlighted—the small number of authors for such a large database study and the lack of ethics committee approval, from which the data collection and analyses were, according to the authors, “deemed exempt”. Following an expression of concern on 3 June, three of the authors retracted the paper. They were unable to complete an independent audit of the data underpinning their analysis and concluded that they “can no longer vouch for the veracity of the primary data sources.”
The puzzle
The responses to the first two papers were warm, often enthusiastic, with some skeptical reservations. Appropriate doubts emerged only when the third appeared. Why the delay?
It was Australian researchers who first pointed out the problems with the third paper, and, apart from two patients, Australian data were not apparently involved in the first and second. Could that be relevant? But there is something else. The first two papers reported benefits, fulfilling everyone’s hopes for effective interventions. The third reported harms and no benefits, dashing the hopes of all those who were expecting the latter.
Realistically we should pay equal attention to benefits and harms, and gauge the balance between them. But do we prefer to believe beneficial outcomes and not harmful ones?
Jeffrey Aronson is a clinical pharmacologist, working in the Centre for Evidence Based Medicine in Oxford’s Nuffield Department of Primary Care Health Sciences. He is also president emeritus of the British Pharmacological Society.
Competing interests: None declared.
