On 9 September, Boris Johnson, the UK’s prime minister, shared his ambitions for using new types of coronavirus tests. “In the near future we hope to start using testing to identify people who are negative, who don’t have coronavirus, who are not infectious…. it should be possible to deploy these tests on a far bigger scale than any country has yet achieved, literally millions of tests, being processed every single day.” [1] Papers revealing the details of this “Operation Moonshot” were leaked around the same time. [2] Liverpool is the first place for this ambition to meet with reality. So what can we learn from this example of rapid policy development and implementation, including as it does an unvalidated test applied on a mass scale for an unclear and shifting purpose?
In August, Oxford University and Public Health England (PHE) at Porton Down were commissioned to evaluate several lateral flow tests for SARS-CoV-2.[3] The results for the Innova test were shared with the government at the end of October, and an immediate decision was made to use it in a pilot of asymptomatic mass testing in Liverpool. The city was given a week’s notice to establish this, aided by the army. The purpose according to the City Council’s website at the time was “to demonstrate that massive asymptomatic testing can help identify far more cases and break the chain of transmission of coronavirus.” It was unclear what would count as “far more,” or how testing would break the chain of transmission.
On 8 November a preliminary report from Porton Down and Oxford was published. [4] It contained little data on the test’s performance among people who were asymptomatic, and made plain that, in everyday use in the field, it might miss one case in two. In one of the batches of testing kits, 16% were found to be unusable.
The government’s decisions—to forge ahead with the pilot and to procure enough tests to roll out mass testing across the country—were made in the full knowledge of the report’s findings. This is hard to understand. Did the dynamics within No 10 make it impossible for health and science advisers to challenge those taking the decision? Was the Innova test the best, or merely the first, of the rapid tests evaluated? And was it really “good enough” for a population-based programme as complex and costly as was now being pushed forward, especially given the prime minister’s earlier emphasis on using mass testing to identify people who are not infected or infectious?
The public health practitioners and academics in Liverpool now faced two challenges. Firstly, how best to deploy this new resource in ways most likely to result in net benefit to the population. Secondly, how to evaluate the approach taken. There was a third challenge: the test itself required further validation in the field. It had, and still has, no approval from the MHRA or WHO for use in this setting: the manufacturer’s instructions specify that it should not be used on asymptomatic people, and nor should the tests be performed by members of the public on themselves. Participants would need to have their results confirmed by a PCR test.
Normally, unapproved use of tests is confined to research settings. This strand of the evaluation—how well the test works in mass testing—looks to have been the responsibility of PHE. The briefest of summary results were reported in the “Community Testing Guide,” published on 30 November. [5] It is unclear why the normal structures of research governance (a protocol, ethical approval, consent etc) were not required. Evaluation of a test not yet in routine use cannot be considered a service evaluation as it requires individuals to undergo additional procedures (the additional PCR tests). The Health Research Authority (HRA) has as yet made no public comment on this aspect of the pilot.
The University of Liverpool is leading what they describe as “service evaluation.” [6] But this description raises further questions. For example, current estimates of the value of repeat testing to enable shorter self-isolation are based on modelling. Can empirical data improve these? And can we better understand the contribution of asymptomatic infection to overall transmission?
The people working within the public health system in Liverpool have done sterling work lending shape to an already very expensive and potentially chaotic intervention. The operational plan is still being created, and the purposes of testing are being refined in real time. [7] The most recent public statement of purpose was by Susan Hopkins, the test and trace medical adviser, on the BBC radio 4 Today programme, when she said that the purpose was case finding. But that was not what the public or participants were told. The pandemic emergency obviously requires processes to be accelerated, but not at any cost, either financial or in terms of public trust. The government has apparently bypassed two bodies central to the reputation of UK science, the MHRA and the HRA, not to mention others with relevant expertise, such as the UK National Screening Committee.
The rush to start the pilot has made it difficult even to measure benefits and harms systematically, let alone compare this initiative with others that might reduce the burden of covid-19 on the population more cost effectively. The normal need to assess opportunity cost has lost all meaning, as the decision to spend this money was made by central government. Liverpool was presented with a resource they had to use. The plan now is to target the asymptomatic testing programme towards “test to release,” “test to enable” and “test to protect”—so called SMART, (Systematic, Meaningful Asymptomatic Repeated Testing), aimed at allowing people who test negative, for example, to return to work, visit older relatives, or stay safe if vulnerable. This rests on an assumption that the test is suitable for these purposes. The high false negative rate suggests that it is not. That each of these three SMART uses of testing will involve other interventions—a PCR test or PPE—raises the question of whether the Innova test is even necessary: it is clearly acknowledged not to be on its own sufficient. Its marginal benefit in Liverpool will now be very hard to assess.
We urgently need to know the original purpose of this initiative. We need to know its cost, especially the cost per case detected or death averted, compared with interventions such as properly run contact tracing and support for self-isolation. And we need the MHRA and HRA to step up and tell us what questions they want answered before further roll out is pursued. Without this basic level of scrutiny and oversight, the prime minister’s lofty ambitions may turn into yet another huge and harmful waste of public money.
Mike Gill, former Regional Director of Public Health, South East England.
Competing interests: None declared.
References:
- https://www.gov.uk/government/speeches/pm-press-conference-statement-9-september-2020 (accessed 6/12/20)
- www.bmj.com/content/370/bmj.m3520?ijkey=e2d550fb879bf701deaad1e4a8dae0b36b1bd3f8&keytype2=tf_ipsecsha (accessed 6/12/20)
- www.gov.uk/government/publications/assessment-and-procurement-of-coronavirus-covid-19-tests/protocol-for-evaluation-of-rapid-diagnostic-assays-for-specific-sars-cov-2-antigens-lateral-flow-devices (accessed 6/12/20)
- www.ox.ac.uk/sites/files/oxford/media_wysiwyg/UK%20evaluation_PHE%20Porton%20Down%20%20University%20of%20Oxford_final.pdf (accessed 6/12/20)
- https://www.gov.uk/government/publications/community-testing-explainer/community-testing-a-guide-for-local-delivery#what-the-community-testing-programme-is (accessed 6/12/20)
- https://www.liverpool.ac.uk/coronavirus/research-and-analysis/covid-smart-pilot/ (accessed 6/12/20) 7. https://soundcloud.com/bmjpodcasts/calum-semple-the-efficacy-of-mass-testing-in-liverpool?in=bmjpodcasts/sets/the-bmj-podcast (accessed 6/12/20)