Public trust in the data about covid-19 is important when entire populations are subject to extended lockdown, financial hardship and economic disruption. Concerns about the accuracy and timeliness of the data on cases and deaths have prompted calls for greater transparency to inform the public, even to accusations that some countries may be hiding the truth for political reasons. 

National trends in the numbers of patients with covid-19 are hard to interpret. This is partly because the severity ranges from rapidly lethal disease to much less severe forms that may be indistinguishable from influenza or the common cold. 

Projections of the likely numbers of cases or deaths, even over a few weeks or months, depend on models that are only as good as the data and the assumptions on which they are based. However, the data have often been flawed, because systematic testing for covid-19 is still rare, and the mortality data are not comprehensive. The assumptions in the models can be difficult to evaluate. Uncertainty around the projected numbers of cases or deaths has thus been wide. Projections have often been substantially revised within a few days. Such projections can be a poor guide to public health policy, as experience in England has shown.

Few countries have tested enough people to obtain a reliable estimate of the proportion of their population that is currently infected, or has recently been infected. The true number of new cases of covid-19 arising per 100,000 population per day or per week (the incidence rate) is thus unknown. Incidence is likely to be substantially under-estimated, because only the more severe cases tend to be formally diagnosed, typically those admitted to hospital. These are the tip of the iceberg. Graphs that show the cumulative number of confirmed cases can be used to assess doubling times—every two, three or four days, etc—and the point at which the curve begins to reach a plateau. Many countries show similar trajectories, but differences in the definition of disease and in the intensity of testing make international comparisons difficult.

Trends in the numbers of deaths from covid-19 are also hard to interpret. In the UK and some other countries, deaths that occur outside hospital, whether in care homes or the community, have only recently been included. If testing is not done in care homes, covid-19 is not likely to appear on the death certificate. One report suggests that half the deaths from covid-19 in five European countries occur outside hospitals, mainly in care homes. [1] The numbers of deaths reported within the previous 24 hours are also subject to time lags in reporting. Finally, most deaths related to covid-19 occur among elderly patients, for whom other serious illnesses such as cancer, diabetes or heart disease may contribute to the sequence of events leading to death. This creates doubt as to whether the underlying cause of death was covid-19. The number of deaths that can be firmly attributed to covid-19 is thus subject to great uncertainty.

The widely reported “case-fatality rates”, ranging from less than 1% up to 8% or more, are not case-fatality rates at all. They are usually derived from the number of deaths attributed to covid-19 during a given period, such as a week, in an entire country, expressed as a proportion of the number of patients with a confirmed diagnosis, but some of those who died will have been diagnosed in previous weeks. If deaths in care homes and the community are excluded from the calculation, the lethality of coronavirus will be under-estimated. Conversely, if patients with mild or asymptomatic disease are excluded, lethality will be over-estimated.

The public will readily understand a case-fatality rate as the proportion of a defined group of patients with confirmed disease who die within a stated period after diagnosis, such as 30 days. Crucially, this requires that all the patients with confirmed disease have been followed up for at least 30 days. Patient groups could be defined on agreed clinical criteria, such as the need for ventilation. That would improve the international comparability of 30-day case-fatality rates among patients with severe disease. If clinical trials were to show efficacy of a treatment that could be quickly and widely introduced, trends in 30-day case-fatality rates would provide a rapid evaluation of its public health impact. Governments should find that useful.

In Italy, the Ministry of Health has compared the daily number of deaths throughout 2020 with the corresponding average in the five previous years in 19 cities with a combined population of 10 million (17% of the national population). [2] The numbers include all deaths, not just those certified as due to covid-19. The data are not nation-wide, but the excess mortality during the covid-19 epidemic is striking, with huge differences between the north and the south, between cities, between broad age groups, and over time. [3] This approach avoids the difficulty in deciding the cause of death, and it provides timely and relevant public health information based on reliable data.

In the UK, the Department of Health and Social Care, NHS England, Public Health England and the Office for National Statistics (ONS) have published five different numbers for deaths from covid-19 in England and Wales. [4] These numbers have a two-fold range. That does not inspire confidence in their accuracy or relevance. The numbers have differed on whether the deaths occurred on a given day or were just reported that day, whether the patients had tested positive for covid-19, whether the death certificate mentioned covid-19, and whether the death occurred in or out of hospital. One recent report suggests that only 7% of covid-19 deaths occur outside hospital in England and Wales, but in Belgium, Ireland, Italy, France and Spain, the estimated range is 42-57%. [1,4]

A more reliable way to assess the public health impact of covid-19 would be to compare the number of deaths from any cause—not just deaths where the certificate mentions covid-19—that have occurred on each day since 1 January 2020 with the average number of deaths on the same day during, say, 2015 to 2019. The number of extra deaths per day in 2020 (the excess mortality) could be presented as the percentage of the corresponding number in previous years. This would provide a guide to the impact of the epidemic on a daily basis. This approach was used to examine excess mortality during the five-day heatwave in England and Wales in 1995. [5] It showed that national mortality during those five days was 9% higher than in previous years, and 16% higher in Greater London.

In the UK, and in some other countries, this approach would provide robust daily figures for excess mortality during the covid-19 epidemic by age and sex, as well as by region. It uses reliable, real-world data, and it includes all deaths. Most deaths in England and Wales are reported within 3-7 days. This will avoid confusion about the date of death, the cause of death and the place of death. Statistical models and assumptions are not required. 

ONS began to do this from 14 April 2020, with a time-lag of only 11 days. [6,7] Weekly mortality is now more than double that in previous years, but the number of extra deaths is much greater than the number of deaths where covid-19 is mentioned on the death certificate. This confirms that the impact of the epidemic cannot be safely assessed from the cause(s) of death on death certificates.

The public health impact of major epidemics such as covid-19 should be based on excess mortality from all causes of death, comparing the number of deaths that occur each day, or each week, with the corresponding number in previous years. This will also facilitate international comparisons.

Michel P Coleman, Cancer Survival Group, Department of Non-communicable Disease Epidemiology, London School of Hygiene and Tropical Medicine

Veronica Di Carlo, Cancer Survival Group, Department of Non-communicable Disease Epidemiology, London School of Hygiene and Tropical Medicine

John R Ashton, Visiting Professor of Applied Public Health, Faculty of Public Health and Policy, London School of Hygiene and Tropical Medicine

Allyson Pollock, Institute of Population Health Sciences, Faculty of Medical Sciences, Newcastle University

Melissa Matz, Cancer Survival Group, Department of Non-communicable Disease Epidemiology, London School of Hygiene and Tropical Medicine

Claudia Allemani, Cancer Survival Group, Department of Non-communicable Disease Epidemiology, London School of Hygiene and Tropical Medicine

Competing interests: None declared

References:

1. Comas-Herrera A, Zalakain J. Mortality associated with COVID-19 outbreaks in care homes: early international evidence. London: International Long-Term Care Policy Network, 12 April 2020. https://ltccovid.org/2020/04/12/mortality-associated-with-COVID-19-outbreaks-in-care-homes-early-international-evidence/?subscribe=success#blog_subscription-3 (accessed 14 April 2020).
2. ISTAT. Resident population. Rome: Istituto Nazionale di Statistica (ISTAT), 2018. http://dati-censimentopopolazione.istat.it/Index.aspx?lang=en (accessed 13 April 2020).
3. Davoli M, De Donato F, De Sario M, et al. Andamento della mortalità giornaliera (SiSMG) nelle città italiane in relazione all’epidemia di Covid-19: terzo rapporto 1 Febbraio – 4 Aprile. Rome: Dipartimento di Epidemiologia, Servizio Sanitaria Regionale Regione Lazio; Centro Nazionale Prevenzione Controlle Malattie, Ministero della Salute, 2020.
4. Office for National Statistics. Deaths registered weekly in England and Wales, provisional: week ending 27 March 2020. Provisional counts of the number of deaths registered in England and Wales, including deaths involving the coronavirus (COVID-19), by age, sex and region, in the latest weeks for which data are available. Newport: ONS, 2020. https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/deaths/bulletins/deathsregisteredweeklyinenglandandwalesprovisional/weekending27march2020 (accessed 14 April 2020).
5. Rooney C, McMichael AJ, Kovats RS, et al. Excess mortality in England and Wales, and in Greater London, during the 1995 heatwave. J Epidemiol Comm Hlth 1998; 52: 482-6. 
6. Office for National Statistics. Deaths registered weekly in England and Wales, provisional: week ending 3 April 2020 Provisional counts of the number of deaths registered in England and Wales, including deaths involving the coronavirus (COVID-19), by age, sex and region, in the latest weeks for which data are available. Newport: ONS, 2020. https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/deaths/bulletins/deathsregisteredweeklyinenglandandwalesprovisional/weekending3april2020 (accessed 14 April 2020).
7. Office for National Statistics. Deaths registered weekly in England and Wales, provisional: week ending 17 April 2020. Provisional counts of the number of deaths registered in England and Wales, including deaths involving the coronavirus (COVID-19), by age, sex and region, in the latest weeks for which data are available. Newport: ONS, 2020. https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/deaths/bulletins/deathsregisteredweeklyinenglandandwalesprovisional/weekending17april2020 (accessed 28 April 2020).