Is the right kind of modelling guiding time-sensitive policy decisions on covid-19?
Is there scientific consensus about how to deal with the autumn surge in coronavirus cases? One could answer yes and no—and be right on both counts. Yes, there is certainly consensus among the groups that constitute the Scientific Pandemic Influenza Group on Modelling (SPI-M), whose projections underwrite policy decisions issued by the UK government. Yes, there are consensus estimates of fundamental epidemiological constants such as the reproduction ratio, or R number. And there is consensus about the fundamental difference between projections of unmitigated outcomes offered by the SPI-M and the predictions of mitigated outcomes based upon less orthodox approaches like dynamic causal modelling (DCM), which we work on at UCL. [1]
However, this consensus dissolves when it comes to model and data selection used to license various policies. This issue has been foregrounded by the decision to go into a national lockdown in England on the basis of SPI-M projections of unmitigated outcomes. Some commentators are now asking if this is the right kind of quantitative modelling to guide decisions that affect the health, wealth, and wellbeing of so many people? Let us examine the recent English lockdown as an example.
The motivation for going into lockdown was to get the R number below one. At the end of October, the SPI-M consensus estimate of R was between 1.1 and 1.3. With these levels, unmitigated daily fatalities were forecast by the SPI-M to be in the thousands, with escalating death rates into the weeks ahead. This stands in stark contrast to the predictions based on the DCM predictions of mitigated outcomes. These suggested that R was below one when lockdown was announced—and the prevalence of infection had peaked several weeks earlier—predicting fatalities in the hundreds that will now plateau. What sets these two sorts of models apart?
SPI-M estimates of the R-number are based upon curve fitting to recent data, while DCM uses these data to infer the prevalence of infection and its instantaneous change. This means SPI-M estimates are about two weeks behind the curve—and vary depending upon the data used for estimation. However, a very different picture emerges under dynamic causal modelling; namely, that we have just reached peak fatalities, weeks before lockdown could have any effect.
DCM has consistently predicted peak fatalities in early November, albeit with varying amplitudes. In the summer, the peak death rate was predicted to be 150 deaths per day, this came down to around 50 per day a month ago and then increased to 100 or so a day, as more data became available. [2] Crucially, the predicted timing of the peak did not change (around the 8th November). This is important because DCM is trying to predict the timing of fluctuations and their responses to mitigation. This aspect of epidemic forecasting is unavailable to SPI-M models that do not predict mitigating population responses. One might ask why is timing so important?
Timing is important for many reasons. For example, if the DCM predictions prove correct, it suggests that the national lockdown has been implemented “after the horse has bolted.” In other words, R had already been suppressed by existing national circuit breakers, fire breakers and regional tier systems. [3] This means that we have less time than the SPI-M forecasts would suggest for restructuring and enhancing local contact tracing. Current DCM predictions suggest there will be a window of opportunity in December, when enhanced test, trace, and supported isolation will make a difference. However, this window of opportunity will disappear almost as soon as it emerges (because cases are predicted to rise again, portending a tertiary wave next spring).
There are many other implications of getting the timing right. One prescient example is the role of school closures. On the basis of SPI-M forecasts, closing schools should make a substantive difference by bringing R down. However, if R is already below one, there is no rationale for closing schools. In turn, this means that we can focus on more pressing issues, such as making schools safe places to work and study.
In short, it is all in the timing. If true, one has to ask whether the kind of modelling pursued by SPI-M is apt for guiding time-sensitive policy decisions—and whether we have confused consensus with orthodoxy. We also have to ask whether the DCM currently used is fit for purpose. Its predictions of mitigation and unlocking after the first wave were accurate to within days. [4] If this predictive validity holds for the current wave, we might see a shift in consensus on which modelling to use. If, on the other hand, we see an escalation of fatalities in the next week or two, the orthodox view was right—and we will have to go back to the drawing board. Time will tell—and time is of the essence.
Karl J. Friston, Scientific Director: Wellcome Centre for Human Neuroimaging. Professor, Queen Square Institute of Neurology, University College London. Honorary Consultant, The National Hospital for Neurology and Neurosurgery.
Anthony Costello, UCL Institute for Global Health, University College London. Professor of Global Health and Sustainable Development. VP: Research.
Karl Friston and Anthony Costello are both members of Independent SAGE (ISAGE). They declare no further conflicts of interest or competing interests.
Note: The quantitative predictions in this opinion piece are based upon dynamic causal modelling. Although this model has been optimised using Bayesian model comparison over the course of the pandemic, it is only one model. As such, any predictions or assertions should be qualified by the fact that they are entirely conditioned upon the model used, which may or may not be the best model.
Footnotes:
1] DCM uses variational Bayes to estimate the unknown parameters of models and then to assess the evidence for alternative models of the same data. It allows for heterogeneity in exposure, susceptibility, and transmission
2] DCM is effectively a data assimilation scheme that continuously adjusts its parameters with incoming data. The particular parameters accounting for the progressive increases in peak death rates (currently around 300 per day) control the rate at which the exposed population enlarges, as the virus spreads into new communities.
3] DCM further predicts that the current lockdown will either be relaxed sooner rather than later—or at least compliance will wane as the prevalence of infection falls.
4] See ‘Posthoc evaluation of model predictions’ section in https://wellcomeopenresearch.org/articles/5-89