The authors argue that uncertainty is inevitable in pandemics and offer some simple rules for better decision making in such circumstances

The covid-19 pandemic is maturing, but uncertainties continue to multiply both for individuals and policy makers. Should I return to work? Should I visit relatives? Which businesses should re-open? What about schools and universities? 

This article is not about the answers to those questions. It is about uncertainty, and how we handle it at both personal and policy levels when urgent action is essential.

Science is sometimes depicted as a methodical, painstaking search for truth, and good policymaking as the translation of those evidence-based truths into action. In pre-pandemic times, these assumptions sometimes (though not always) held. But the complexity of science and policymaking in the context of uncertainty has been brought into sharp focus by this pandemic. [1] 

The disease is novel. Some recent research findings can probably be given the status of facts, but overall, the evidence base on effectiveness of interventions—both preventive and therapeutic—remains patchy and limited. The extent to which research findings from other diseases (and even other coronaviruses) can be extrapolated to covid-19 is contested. 

As each country’s covid-19 experience shifts from an acute national disaster to a chronic policy crisis, we all—clinicians, scientists, policymakers, and citizens—need to move on from imagining that the uncertainties can be resolved. They may never be. 

This is because covid-19 is, par excellence, a complex problem in a complex system. [2] Complex systems are, by definition, made up of multiple interacting components. Such systems are open (their boundaries are fluid and hard to define), dynamically evolving (elements in the system feed back, positively or negatively, on other elements), unpredictable (a fixed input to the system does not have a fixed output) and self-organising (the system responds adaptively to interventions). Complex systems can be properly understood only in their entirety; isolating a part of the system in order to “solve” it does not produce a solution that works across the system for all time. Uncertainty, tension and paradox are inherent; they must be accommodated rather than resolved. [3]

In circumstances like this, uncontested facts—things that are ascertainable, reproducible, transferable and predictable—tend to be elusive. Most decisions must be based on information that is flawed (imperfectly measured, with missing data), uncertain (contested, perhaps with low sensitivity or specificity), proximate (relating to something one stage removed from the real phenomenon of interest) or sparse (only available for some aspects of the problem). [4] 

Data that are trustworthy, certain, definitive, and plentiful can be presented as facts and evidence based decisions can follow from them. These are the data we hope for and search for; the science that will inform the ultimate exit strategy from this pandemic. [5]  But the stage of the current pandemic requires us to work with the kinds of imperfect data described above, so different approaches are needed. [4] 

All of us making use of such data should be aware of our own confirmatory biases, avoiding groupthink and applying the same standards of scrutiny to findings that appear to support our prior beliefs or personal biases as to those which challenge them.  In such circumstances, we all may need to make decisions based on “balance of probabilities” rather than “evidence beyond reasonable doubt”, and consider how it meshes with existing interpretations, values and priorities. [6] 

Instead of seeking—or feigning—certainty, we should be open about uncertainty, and transparent in the ways in which we acknowledge the limitations of the imperfect data we have no choice but to use. Teams should be encouraged to admit ignorance, explore paradoxes and reflect collectively. [7] This will improve the quality of decision-making by supporting constructive scrutiny and make us more open to revising our decisions as new data and evidence emerge. 

Even when an evidence base seems settled, different people will reach different conclusions based on the same evidence. When the evidence base is at best inchoate, divergences will be greater. When epistemic conflicts remain unacknowledged and suppressed, they can be destructive. But if surfaced and debated, competing interpretations can help us productively to accept all options as flawed, and requiring negotiation between a range of actors in the complex system. [8] If there is mutual respect and space for negotiation, such conflicts can be channelled into multifaceted solutions and adaptive actions. [9] 

We may all face the same pandemic, but our knowledge, worldviews, and values differ. Rather than demonising others for their alternative interpretations, we should celebrate the different perspectives that those who engage rigorously with the science can bring to bear on the unavoidably flawed data we have to work with. In this context, purist pursuit of an illusory one-dimensional truth is doomed to failure. Instead, we must generate and collaborate to achieve “viable clumsy solutions”. By carefully evaluating how these imperfect responses unfold in messy real-world settings, we can help to build the multifaceted evidence base that the world urgently needs. [10] 

Managing uncertainty in a pandemic: five simple rules 

1. Most data will be flawed or incomplete. Be honest and transparent about this.
2. For some questions, certainty may never be reached. Consider carefully whether to wait for definitive evidence or act on the evidence you have.
3. Make sense of complex situations by acknowledging the complexity, admitting ignorance, exploring paradoxes and reflecting collectively.
4. Different people (and different stakeholder groups) interpret data differently. Deliberation among stakeholders may generate multifaceted solutions.
5. Pragmatic interventions, carefully observed and compared in real-world settings, can generate useful data to complement the findings of controlled trials and other forms of evidence.

Harry Rutter, Professor of Global Public Health, Department of Social and Policy Sciences, University of Bath, Bath, UK

Miranda Wolpert, Head of Mental Health Priority Area, Department of Clinical Education and Health Psychology, Wellcome Trust, London, UK

Trisha Greenhalgh, Professor of Primary Care Health Sciences, Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK

Competing interests: None declared

References:

1. Lancaster K, Rhodes T, Rosengarten M. Making evidence and policy in public health emergencies: lessons from COVID-19 for adaptive evidence-making and intervention. Evidence & Policy: A Journal of Research, Debate and Practice 2020
2. Hawe P, Shiell A, Riley T. Theorising interventions as events in systems. American journal of community psychology 2009;43(3-4):267-76.
3. Greenhalgh T, Papoutsi C. Studying complexity in health services research: desperately seeking an overdue paradigm shift: BioMed Central, 2018:95.
4. Wolpert M, Rutter H. Using flawed, uncertain, proximate and sparse (FUPS) data in the context of complexity: learning from the case of child mental health. BMC Med 2018;16(1):82. doi: 10.1186/s12916-018-1079-6 [published Online First: 2018/06/14]
5. Farrar J. This virus isn’t going away. The only way to beat it is to work together. London: Wellcome Trust. Accessed 5th July 2020 at https://wellcome.ac.uk/news/virus-isnt-going-away-only-way-beat-it-work-together?utm_source=twitter&utm_medium=o-wellcome. 2020.
6. Fischer A, Threlfall A, Meah S, et al. The appraisal of public health interventions: an overview. Journal of Public Health 2013;35(4):488-94.
7. Lanham HJ, Leykum LK, Taylor BS, et al. How complexity science can inform scale-up and spread in health care: understanding the role of self-organization in variation across local contexts. Social Science & Medicine 2013;93:194-202.
8. Tsoukas H. Don’t simplify, complexify: From disjunctive to conjunctive theorizing in organization and management studies. Journal of management studies 2017;54(2):132-53.
9. Mouffe C. Agonistics: Thinking the world politically: Verso Books 2013.
10. Ulijaszek SJ. Models of Obesity: From Ecology to Complexity in Science and Policy (Vol. 78). Cambridge: Cambridge University Press 2017.