“Dear sir, I have completely failed to understand a simple criticism of our work, please tell everyone, yours, BBCnews“ Tweet by @bengoldacre, 4 November 2011
The misuse of epidemiology is everywhere. Lets look at two prominent culprits.
”Three fold variation’ in UK bowel cancer death rates’ splashed a recent BBC News headline.
Had someone not thought to check the numbers we’d have all gone away grumbling about postcode lotteries. And we’d all have been wrong. What was going on?
All things vary. If a coin is flipped several times, chance dictates it will not land heads exactly half the time. But the more times you flip the coin, the closer to a half you get.
In the year in question, there were only six deaths from bowel cancer in the Shetland Islands, twelve in Antrim, fourteen in Watford, and so on. Even with a uniform rate of bowel cancer deaths across the UK, might there only be two deaths in Shetland the following year? Would that merit a headline or could it just be statistical noise?
Thankfully, as reported by Ben Goldacre, someone properly analysed the data. The variation in death rates seen was actually less than would be expected by chance. Embarrassingly, the BBC failed to understand the criticism.
Whilst the poor quality of science journalism has been well documented by Ben and others over the years, ineptitude is less concerning than the deliberate misuse of data.
The debate surrounding the recent UN High Level Meeting on Non Communicable Disease (NCD) was mired in epidemiological naughtiness. Among the most prominent culprits were the NCD Alliance, a grouping of NGOs backed by the pharmaceutical industry, who unfortunately – given their funding – emerged as the leading advocacy organisation.
“NCDs were responsible for 63% of all global deaths in 2008. This is not just a statistic, it is the deaths of 36 million people. With the incidence of NCDs predicted to rise by 17% over the next ten years worldwide, we must work together to ensure it is world leaders who attend the Summit and agree to a concrete set of commitments that will result in sustained action.”
Professor Jean Claude Mbanya, President of the International Diabetes Federation, quoted in a press release from the NCD Alliance.
Given immortality is not currently achievable and that we must all die of something, presenting mortality data as counts is thoroughly misleading. Non age standardised counts fail to capture the difference between a death from a respiratory tract infection aged five and a death from ischaemic heart disease aged ninety five.
With huge expertise within the alliance, it is hard to believe that their presentation of data in this manner was anything other than a deliberate choice to spin the data and inflate the contribution of NCDs. Whilst counts are easier to understand than Disability Adjusted Life Years lost or age adjusted rates, the alliance could have presented data on numbers of premature deaths – less misleading and totally comprehensible.
I don’t dispute the assertion that NCDs are emerging as a major problem in low and middle income countries, and I’m gutted the summit achieved so little. However, the data is compelling when presented honestly. It doesn’t need to be spun.
It is very easy to misrepresent epidemiological data and for the unscrupulous to spin data for the press. Given the volume of coverage epidemiological research generates, groups such as the Science Media Centre have their work cut out.
I don’t know what the solution is. Ben Goldacre’s suggestion that we need more science editors and less science reporters seems sensible. I’d be interested in people’s thoughts.
Tom Yates qualified as a doctor in London in 2009. He subsequently completed an academic foundation programme in Oxford, where he was involved in research on hepatitis vaccines. He commenced an MSc in Epidemiology at the London School of Hygiene and Tropical Medicine in September 2011. He is interested in infectious disease and population health. He blogs on epidemiology and population health at sickpopulations.wordpress.com