The current question “to mask or not to mask?” has again unmasked our limitations in interpreting medical research and in handling the associated uncertainty, argues James Brophy

While there is no doubt regarding the physical, mental, and economic carnage due to severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), there is considerable debate regarding the evidence of masks to reduce its spread. Paradoxically, the publication last week of the first randomized trial evaluating masks during the current covid-19 pandemic and a meta-analysis of older trials seems to have heightened rather than reduced the uncertainty regarding their effectiveness. [1,2] Herein I briefly explore this paradox.

The DANMASK-19 trial was performed in Denmark between April and May 2020, a period when public health measures were in effect, but community mask wearing was uncommon and not officially recommended. [1] All participants were encouraged to follow social distancing measures. Those in the intervention arm were additionally encouraged to wear a mask when in public and were provided with a supply of 50 surgical masks and instructions for proper use. Crucially, the outcome measure was rates of infection among those encouraged to wear masks and not in the community as a whole, so the study could not evaluate the most likely benefit of masks, that of preventing spread to other people. The study was designed to find a 50% reduction in infection rates among mask wearers. Among the 4862 participants who completed the trial, infection with SARS-CoV-2 occurred in 42 of 2392 (1.8%) in the intervention arm and 53 of 2470 (2.1%) in the control group. The between-group difference was −0.3% point (95% CI, −1.2 to 0.4%; P = 0.38) (odds ratio, 0.82 [CI, 0.54 to 1.23]; P = 0.33). This led to the published conclusion: “The recommendation to wear surgical masks to supplement other public health measures did not reduce the SARS-CoV-2 infection rate among wearers by more than 50% in a community with modest infection rates, some degree of social distancing, and uncommon general mask use. The data were compatible with lesser degrees of self-protection.” The conclusion initially appears obtuse (what is the unambiguous answer to the question—do masks work or not?) but in reality, is very astute and avoids the common error of banally interpreting a statistically non-significant result as a “negative” trial.

Incorrect interpretation of “negative” trials abounds, even among thoughtful academics, regardless of their personal beliefs about the studied intervention. It arises largely from a misunderstanding of the null hypothesis significance testing (NHST) paradigm used in most medical research. The limitations of this paradigm have long been appreciated by statisticians, but making inroads into the medical community has been challenging despite multiple recent warnings about the dangers. [3-5] The DANMASK-19 trial provides two immediate examples of study misinterpretation. First, while the editorial accompanying the trial concludes that masks may work, it does so while also implying that the trial itself was negative, stating “… despite the reported results of this study, (masks) probably protect the wearer.” [6] And second, an opinion article in the lay press by two esteemed university professors was entitled “Landmark Danish study finds no significant effect for face mask wearers.” [7]. Despite honourable intentions, in my view, both interpretations are incorrect.

The results of DANMASK-19 do not argue against the benefit of masks to those wearing them but actually support their protective effect. If we step outside the statistical straitjacket of NHST, with its inappropriate focus on the null hypothesis and the dichotomania surrounding P = 0.05, it can be shown that these data best support an 18% reduction in infections among mask wearers and find as much evidence for a 33% reduction as for no effect. [8]

Bayesian analysis of the DANMASK-19 trial alone shows an 81% probability of fewer infections among those encouraged to wear a mask and a 35% probability that mask wearers will avoid more than five infections/1000 individuals. Similar results are achieved with a Bayesian analysis that combines the DANMASK-19 results with prior knowledge about masks, expressed as the relative risk observed in the Cochrane review of older randomised trials of masks (RR 0.91 95%CI [0.66, 1.26]). [2]

Before arguing about whether this effect is large enough to support a public policy recommending masks, we should consider several additional elements. First, while accepting the dangers of wild extrapolations, in a country of 300 million people even three fewer infections per 1000 mask wearers could lead to almost one million fewer infections. Second, remember that the DANMASK-19 trial did not consider long term infection rates, but measured outcomes after only one month of the intervention and in a population with a very modest infection rate. Imagine an open cohort, where infected participants would be replaced by new uninfected participants, and where the DANMASK-19 one month rate difference is continually observed over the entire nine months of the current pandemic. Under this scenario, we might expect to observe 36 (95%CI 12 -57) fewer infections per 1000 mask wearers over the duration of the pandemic. Of course, if the baseline infection rate is higher, it is likely that the number of infections prevented would increase proportionally. Finally, the economic costs and adverse effects of masks are trivial, especially compared with drug treatments and even the soon to be marketed vaccines. [9] This is important as while the current evidence is imperfect it appears “good enough” to make policy decisions today, given the absence of any severe downside to the intervention. While the benefit of masks is “not beyond all reasonable doubt” and while future evidence could be incorporated with our current evidence to refine our state of knowledge, further research has not only a direct cost, but also an opportunity cost for other research that won’t be done. In decision theory terms, the expected value of perfect information for mask effectiveness may be sufficiently low that further research into this intervention is perhaps not worth the extra value returned.

Given the estimated 60 million SARS-CoV-2 infections worldwide, the DANMASK-19 trial’s finding of an 18% reduction in the infection rate among mask wearers is of enormous potential public health importance. The theoretical benefits of masks are even larger, given that DANMASK-19 was a pragmatic trial in a low incidence population and that a substantial number of participants did not fully comply with the intervention. Moreover, as has been said, this trial examined only one half of any potential benefit of masks—does it protect the wearer?—and did not consider any possible benefit in reduced transmission of infection to others. 

Are we surprised that the results of this study have been mis-interpreted in both the scientific and lay press as not providing any meaningful benefit, despite a valiant attempt by the DANMASK-19 authors to avoid these misunderstandings? [1] Unfortunately not, as the interpretation and reporting of medical research have long been described as a “scandal.” [10] The current question “to mask or not to mask?” has again unmasked our limitations in interpreting medical research and in handling the associated uncertainty. 

James M Brophy, Professor of Medicine & Epidemiology (McGill University) McGill University Health Center.

Competing interests: There are no relationships with industry. JMB is a research scholar supported by Les Fonds de Recherche Québec Santé.

References:

1. Bundgaard H, Bundgaard JS, Raaschou-Pedersen DET, von Buchwald C, Todsen T, Norsk JB, et al. Effectiveness of Adding a Mask Recommendation to Other Public Health Measures to Prevent SARS-CoV-2 Infection in Danish Mask Wearers : A Randomized Controlled Trial. Ann Intern Med. 2020.
2. Jefferson T, Del Mar CB, Dooley L, Ferroni E, Al-Ansary LA, Bawazeer GA, et al. Physical interventions to interrupt or reduce the spread of respiratory viruses. Cochrane Database Syst Rev. 2020;11:CD006207.
3. Amrhein V, Greenland S, McShane B. Retire statistical significance. Nature. 2019;567:305-7.
4. Wasserstein R, Lazar N. The ASA’s Statement on p-Values: Context, Process, and Purpose. The American Statistician. 2016;70:2:129-33.
5. Wasserstein R, Schirm A, Lazar N. Moving to a World Beyond “p < 0.05”. The American Statistician. 2019;73:1-19.
6. Frieden TR, Cash-Goldwasser S. Of Masks and Methods. Ann Intern Med. 2020.
7. Heneghan C, Jefferson T. Landmark Danish study finds no significant effect for facemask wearers 2020 [Available from: https://www.spectator.co.uk/article/do-masks-stop-the- spread-of-covid-19-.
8. Rafi Z, Greenland S. Semantic and cognitive tools to aid statistical science: replace confidence and significance by compatibility and surprise. BMC Med Res Methodol. 2020;20(1):244.
9. Brophy JM. US purchases world stocks of remdesivir: why the rest of the world should be glad to be at the back of the queue. BMJ. 2020;370:m2797.
10. Altman DG. The scandal of poor medical research. BMJ. 1994;308(6924):283-4.