Access thebmj.com - The BMJ logo

Responding to the “known unknowns” of covid-19

There are two kinds of ignorant people, those who don’t know and know they don’t know, and those who don’t know and don’t know that they don’t know. Hitherto the direction of most medical matters has lain in the hands of this latter class. It can be readily understood that in trying to improve matters their attempts will not be guided by any rational principle, but will be haphazard, and in consequence ineffective. [1] — Sir James Mackenzie 1919

No, it was not the former US Secretary of Defence Donald Rumsfeld who inspired our editorial and linked BMJ webinar “Covid-19: Known Unknowns.” [2-4] It was the pioneering clinical epidemiologist James Mackenzie. [1,2] Maybe you knew who deserved the credit. Or maybe (punchline alert…) you just thought you did. 

In the weeks since that webinar, our collective uncertainty has remained evident—not least in the UK, with the identification of a new covid variant and subsequent (re)lockdown. Germany, lauded in Autumn 2020 for apparently getting everything right, now faces high death rates (and has become strangely invisible to erstwhile fans, with honourable exceptions). [5-7] The regimens for vaccines—the optimal dosing schedule for one of which was discovered only by accident—are hotly debated. The depiction of school children in the “Covid Wars” oscillates between that of victims and perpetrators. Tests have been either our salvation or our end. Amid so many new and continuing uncertainties, one thing at least is clear: the need for more open and respectful discussion. 

From 28 January 2021 a series of topic-specific webinars under the general heading “Covid-19: known unknowns” will run fortnightly, beginning with the touchstone issue of school closures. These events will continue to provide a forum for Mackenzie’s first step to wisdom: recognition of what is uncertain. 

Pandemics offer fertile ground for reflection on fundamental issues about the nature of knowledge. [8] This is for two related reasons. The first is the unintuitive fact that despite the impressively large numbers reported daily—of cases, tests, vaccines delivered, ICU admission and deaths—we are observing a much smaller number of temporally and spatially coherent outbreaks. But it is these aggregate events—not the individuals involved—that are our proper data points.

In foundational experimental studies of controlled epidemics in mice, which were conducted between ~1920-1940 by William Topley and Major Greenwood, “hundreds of thousands of mice [were] sacrificed”, but as “one thousand individuals may provide us with only one group-observation” the “sample we assembled in 20 years was, statistically speaking, a small sample.”[9]  The introduction of infectious mice into the colonies, the degree of overcrowding, household size, the level of pre-existing immunity, nutrition, genetic strain, all (and more) were tightly controlled. Yet, the overwhelming conclusion was that chance events contribute substantially to the generation of distinct epidemic patterns. [10, 11]  Moving from mice to (wo)men, it remains the case that “if you’ve seen one pandemic, you’ve seen … one pandemic.” [12]  

The second reason for reflection is the human tendency to over-explanation (with its technical concomitant, the over-fitting of data to a model). This, when combined with poor or erroneous data, and strong preconceptions, creates a potent mix, reflected in the impression that (on average, of course) the more certain an academic pundit sounds, the further their primary expertise will be from applied infectious disease dynamics. 

This gloomy outlook is not specific to infectious disease and epidemics, but pervades biological (and biomedical) knowledge in general. [13] Indeed, it is a reason for regarding much personalised medicine as hype. [14]  But we must not despair. At a group level, we can learn and intervene to great effect. 

Nothing, however, can be learnt from the arbitrary, haphazard, and confusing imposition of different interventions, whether “world beating” or otherwise. Interventions—such as population testing, vaccines and vaccination schedules, mask mandates, school closures—and even lockdowns—could all be introduced in a manner which allows for the gathering of  evidence as to their effectiveness. NHS records can be linked at impressive speed, providing outcome data for monitoring the effects of such interventions. [15] To exploit this potential, we suggest two requirements. Firstly, a system which allows for rapid deployment of the tools required for establishing the informative introduction and follow-up of interventions (such as through stepped-wedge designs). [16] This should be a priority before the next pandemic arrives. Secondly, an acknowledgment that the outcomes are uncertain, as this is the spur to acquiring knowledge, and moving into the domain of the known knowns.      

Details of the content and (necessary, but free) registration for the forthcoming series of covid webinars is here. Topics will include vaccinations, new variants of SARS-CoV-2, testing, the balance of benefits and harms of interventions, and the “zero covid” strategy. However, things are changing rapidly and often unexpectedly, thus we will not plan too far in advance, and you can suggest topics for future webinars on the website. We hope this series will encourage agreement with James Mackenzie that acknowledging unknowns is the first step towards advancing knowledge. 

George Davey Smith, Professor in Clinical Epidemiology, MRC Integrative Epidemiology Unit, University of Bristol, Bristol, UK

Michael Blastland, Writer and broadcaster, Winton Centre for Risk and Evidence Communication, Centre for Mathematical Sciences, University of Cambridge, Cambridge, UK

Allyson M Pollock, Professor in Public Health, Population Health Sciences Institute, Faculty of Medical Sciences, Newcastle University, Newcastle, UK

Competing interests: We have read and understood BMJ policy on declaration of interests and declare that in addition to the COI statement in our earlier editorial a further thing GDS and MB have got wrong is in the editorial we stated with respect to projections of the mortality consequences of an unmitigated covid epidemic in the UK that: “The shifting denominator between Great Britain and the United Kingdom used when communicating this figure would in itself make a difference of considerably more than 10,000.” This is incorrect: the difference would be around 10,000, not “considerably more.” AP was a member of independent SAGE.

References:

  1. Mackenzie J. Letter to the Editor: Defects due to want of knowledge. Medical Research Times (London, England). 1919;26:42267:8. Reprinted in Int J Epidemiol 2016;45:1746-1747.
  2. Davey Smith G. Commentary: Known knowns and known unknowns in medical research: James Mackenzie meets Donald Rumsfeld. International Journal of Epidemiology. 2017;45(6):1747-8.
  3. Davey Smith G, Blastland M, Munafò M. Covid-19’s known unknowns. BMJ. 2020;371:m3979.
  4. COVID 19: Known Unknowns (Facing up to scientific uncertainty during a pandemic). BMJ Webinar 20th November 2020.
  5. Han E, Tan MMJ, Turk E, Sridhar D, Leung GM, Shibuya K, et al. Lessons learnt from easing COVID-19 restrictions: an analysis of countries and regions in Asia Pacific and Europe. The Lancet. 2020;396(10261):1525-34.
  6. Chivers T. The German coronavirus mystery: UnHerd; 14th October 2020 [updated 14th October 2020. Available from: https://unherd.com/2020/10/can-britain-learn-from-germanys-covid-success/.
  7. Chivers T. What explains the Germany Covid Surge? UnHerd.  14th October 2021 [Available from: https://unherd.com/thepost/what-explains-the-german-covid-surge/.
  8. Morens DM, Taubenberger JK. Pandemic influenza: certain uncertainties. Rev Med Virol. 2011;21(5):262-84.
  9. Greenwood M. William Whiteman Carlton Topley, 1886 – 1944. Obituary Notices of Fellows of the Royal Society. 1944;4(13):698-712.
  10. Greenwood M, Hill A, Topley W, Wilson J. Experimental Epidemiology. Medical Research Council Special Report Series no 209, London: His Majesty’s Stationery Office, . 1936.
  11. Topley W. The Croonian Lecture: The Biology of Epidemics. Proceedings of the Royal Society of London Series B, Biological Sciences. 1942;130(861):337-59.
  12. Kucharski A. The Rules of Contagion: Why Things Spread – and Why They Stop (Wellcome Collection). London: Profile Books Ltd; 2020.
  13. Smith GD. Epidemiology, epigenetics and the ‘Gloomy Prospect’: embracing randomness in population health research and practice. International Journal of Epidemiology. 2011;40(3):537-62.
  14. Davey Smith G. Some constraints on the scope and potential of personalised medicine.  Centre for Personalised Medicine Seminars. https://www.youtube.com/watch?v=uiCd9m6tmt0. University Centre for Personalised Medicine, Oxford, UK2020.
  15. Williamson EJ, Walker AJ, Bhaskaran K, Bacon S, Bates C, Morton CE, et al. Factors associated with COVID-19-related death using OpenSAFELY. Nature. 2020;584(7821):430-6.
  16. Smith P, Morrow R, Ross D. Field Trials of Health Interventions: A Toolbox (3 ed.). Oxford Medicine Online: Oxford University Press; 2015.