{"id":50461,"date":"2021-06-15T14:09:52","date_gmt":"2021-06-15T13:09:52","guid":{"rendered":"https:\/\/blogs.bmj.com\/bmj\/?p=50461"},"modified":"2021-06-24T07:42:52","modified_gmt":"2021-06-24T06:42:52","slug":"sheila-bird-diagnostic-tests-must-be-more-rigorously-regulated","status":"publish","type":"post","link":"https:\/\/blogs.bmj.com\/bmj\/2021\/06\/15\/sheila-bird-diagnostic-tests-must-be-more-rigorously-regulated\/","title":{"rendered":"Sheila Bird: diagnostic tests must be more rigorously regulated"},"content":{"rendered":"

Study designs, regulation, and transparency of in-vitro diagnostic tests matter and need to be embedded in pandemic planning, argues Sheila M. Bird<\/p>\n


\nA <\/span>recent report from the Royal Statistical Society\u2019s Working Group<\/span><\/a> on Diagnostic Tests has called for new standards for diagnostic tests to assure the performance of future tests. The report makes 22 recommendations on study-design; regulation; and transparency. The Royal Statistical Society (RSS) convenes working parties infrequently, usually only when a major misuse of statistics needs correction, so it is noteworthy that the RSS has issued a report on this topic. [<\/span>1-5]<\/span><\/p>\n

Since the early 1980s, stricter regulations apply in respect of in-vitro diagnostic tests for infectious diseases which could contaminate the blood supply. Despite the global pandemic, and an initial infection-fatality-rate of around 1%, SARS-CoV-2 does not threate<\/span>n the<\/span> blood supply and tests for it are not held to a common statistical benchma<\/span>rk as used<\/span> for drugs or vaccines. On 10 June 2021, the <\/span>US Food and Drugs Administration (FDA) issued a safety communication<\/span><\/a> warning the public to stop using the Innova SARS-CoV-2 antigen rapid qualitative test over concerns about performance of the test. A statement from the UK\u2019s regulator (Medicines & Healthcare products Regulatory Agency MHRA) is expected as the Innova lateral flow antigen tes<\/span>ts have been in wide use in the UK where the supply of the devices is only permitted by the Department for Health and Social Care.\u00a0\u00a0<\/span><\/p>\n

Antigen tests for SARS-CoV-2 rely primarily on a nasopharyngeal swab and manufacturers are able to apply for emergency use authorization on the basis of self-certification, so-called Conformit\u00e9 Europ\u00e9ene (CE), with a similar process in the United States. Tests can be used in contexts which differ from<\/span> their C<\/span>E-use. For example, in the UK, the deployment of the Innova lateral flow antigen test for mass screening of asymptomatic people in Liverpool in November 2020 deviated from its CE-use, which was for people with symptoms of covid-19. The rollout of mass screening in the UK was presaged only by modest investigation in asymptomatic persons.<\/span><\/p>\n

Crucially, in Liverpool <\/span>a dual-swabbing study-design<\/span><\/a> was used whereby 6,000 asymptomatic people agreed to provide contemporaneous swabs for th<\/span>e Innova r<\/span>apid test (result within 30 minutes) and to be tested using quantitative reverse transcription Polymerase Chain Reaction tests (qRT-PCR). qRT-PCR tests are considered to be the UK\u2019s gold standard test for diagnosing SARS-CoV-2 infection in symptomatic people. Associated with qRT-PCR is an amplification cycle-threshold (ct-value), which is lower when viral load is higher.<\/span><\/p>\n

Liverpool\u2019s <\/span>study-design<\/span><\/a> revealed tha<\/span>t the Innova l<\/span>ateral flow test diagnosed only<\/span> 28 (40%) of the 70 infections [95% CI: 28% to 52%]<\/span><\/a> that were identified using qRT-PCR, but the rapid lateral flow test performed better at diagnosing \u201cinfectiousness\u201d, that is: infections for which ct-value was below 25 (67%) [95% CI: 41% to 87%].<\/span><\/p>\n

In March 2021, the UK government rolled ou<\/span>t assisted-screen<\/span>ing of secondary school pupils in England for SARS-CoV-2 on their return to school (three times in two weeks). This was another context-of-use for the Innova lateral flow antigen test due to pupils\u2019 younger age and the substantially lower prevalence of asymptomatic infection in the community compared to November 2020, when mass testing was rolled out in Liverpool.\u00a0<\/span><\/p>\n

Not only was there no <\/span>properly-designed evaluation <\/span>of <\/span>this mass screening in schools<\/span><\/a>, but <\/span>confirmation using qRT-PCR tests<\/span><\/a> for positive lateral flow test results was also suspended, despite <\/span>the RSS warning <\/span>that<\/span><\/a>, due to low prevalence of asymptomatic infection in the community, half of the lateral flow test results would be negative when adjudicated<\/span>. [6]<\/span><\/p>\n

The UK has learned the hard way that, for diagnostic testing, study design matters and we cannot assume that test-performance can be generalised from one context of use to another (symptomatic to asymptomatic; adult to child; high to low prevalence; high viral load versus vaccine-reduced; or across different variants of concern).\u00a0<\/span><\/p>\n

The recent RSS report on new standards for diagnostic tests notes that robust studies of <\/span>analytical performance<\/span><\/i> provide necessary, but insufficient evidence to impleme<\/span>nt <\/span>in vitro<\/span><\/i> d<\/span>iagnostics. Confidence intervals for the sensitivity (% of infected persons who are correctly detected by the test) and specificity (% of uninfected persons correctly labelled by the test as uninfected) for each intended use should be standard practice, not exceptional. Lesser sensitivity and specificity can be adjusted for in surveillance studies. [7] Study designs for direct comparison of competing <\/span>in vitro<\/span><\/i> diagnostic tests should be a priority in the planning for future pandemics, which requires:<\/span><\/p>\n

    \n
  1. Identification of multi-site networks to recruit patients or citizens willing to provide relevant biological specimens (including serially);<\/span><\/li>\n
  2. Creation and maintenance of specimen banks;<\/span><\/li>\n
  3. Multi-disciplinary input\u2014statistical and regulatory<\/span>\u2014<\/span>as well as clinical and laboratory, to agree evaluation strategies;<\/span><\/li>\n
  4. Expedited study-protocol and ethical approvals before access to banked specimens.<\/span><\/li>\n<\/ol>\n

    In vitro<\/span><\/i> diagnostic tests require stricter regulation because mis-represented or mis-applied tests can be unsafe<\/span>\u2014<\/span>as alerted by FDA warning <\/a><\/span>last<\/span> week. The RSS looks to the MHRA, the UK\u2019s regulator to take a strong lead to ensure international harmonization on higher standards of diagnostic tests. In particular, the RSS recommends the strengthening of Target Product Profiles<\/span> for each intended use and consideration of the conseque<\/span>nces of a positive test result, which for SARS-CoV-2 ranges from inappropriately liberalised behaviour following a false negative result, to deprivation of liberty via self isolation following a positive result. Both of these outcomes have considerable implications for individuals and the community.\u00a0\u00a0<\/span><\/p>\n

    Stricter regulation helps in achieving greater transparency because protocols for field or clinical evaluation studies would be publicly available, well-scrutinized, ethically approved, and any post-hoc analyses should be clearly identifiable as exploratory. Regulators would also have a crucial role in vetting the information provided to the public about test-performance in different contexts. People should not be confused<\/span>\u2014<\/span>inadvertently or deliberately<\/span>\u2014<\/span>about which test is being offered; and the public needs to be clear about when confirmatory testing is required and why<\/span>\u2014<\/span>including for public health reasons such as genomic tracking.\u00a0<\/span><\/p>\n

    Sheila M. Bird<\/i><\/b>, formerly Programme Leader, MRC Biostatistics Unit, Cambridge<\/span><\/i><\/p>\n

    Competing interests:<\/i><\/b> SMB is member of the Royal Statistical Society\u2019s COVID-19 Taskforce and chairs its RSS\/DHSC Panel on NHS Test & Trace; member since January 2021 of PHE\/NHS Test & Trace\u2019s Testing Initiatives Evaluation Board; proposer and member of RSS Working Group on Diagnostic Tests; past member\/chair of five out of six previous RSS Working Parties in the past 30 years.<\/span><\/i><\/p>\n

    See also<\/strong>:<\/p>\n