Paul Glasziou: How to hide trial results in plain sight

Paul Glasziou discusses why trial results need to be better presented, so that readers can understand and act on the results.

The AllTrials campaign rightly asks for “all trials registered; all results reported,” but I would add that all reported results should be readily comprehensible to readers. Unfortunately, many studies are analysed or presented so poorly that readers find it difficult to find and interpret the key results.[1] This means that many studies can mislead readers. Such communication failure effectively wastes the researchers’ considerable effort in designing, conducting, analysing, and publishing the study—the ultimate purpose being for readers to understand and act on its results.

A recent trial of a 5 versus 10 day course of antibiotics for acute otitis media [2] illustrates this problem well. The trial was a very well done randomised trial of 520 children with carefully diagnosed acute otitis media. The study results were presented in extensive detail—indeed, the blizzard of detail meant that finding and understanding the main results takes considerable detective work by the reader.

Let’s look at those results by piecing together the main patient relevant outcome measure: the symptoms experienced by both groups of children. Doing that requires readers to find and contrast rows in two separate tables (Table 1 and 3), and then mentally compare them. To make that job easier, I’ve graphed these below.

Figure: Graphical version of data on 5 vs 10 days antibiotics, extracted from Table 1 & 3. [2]

When I’ve shown this figure to colleagues their interpretations are “Well, there is no difference,” or “There’s a tiny difference at the end, but most children have gotten better.” Contrast that with the authors’ conclusions, which were: “Among children 6 to 23 months of age with acute otitis media, reduced-duration antimicrobial treatment resulted in less favorable outcomes than standard-duration treatment.” Whatever conclusion—statistical and clinical—you come to, the burden of interpretation is much easier with these results in a figure rather than extracting several numbers on different pages.

This is not a trivial concern. Trial results can influence the clinical decisions made by thousands or tens of thousands of clinicians around the world. So presenting the results in a clear, unbiased, and understandable way is of paramount importance. Editors should insist on clear, simple presentations of the main results—preferably in graphical formats. Without that, authors and editors will continue to contribute to the considerable waste in research and the gaps between research and practice.

Paul Glasziou is professor of evidence based medicine at Bond University and a part time general practitioner.

Competing interests: None declared.

References

[1] Glasziou P, Altman DG, Bossuyt P, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014 Jan 18;383(9913):267-76.
https://www.ncbi.nlm.nih.gov/pubmed/24411647

[2] Hoberman A, Paradise JL,Rockette HE, et al. Shortened Antimicrobial Treatment for Acute Otitis Media in Young Children. N Engl J Med. 2016;375:2446-56.
http://www.nejm.org/doi/full/10.1056/NEJMoa1606043

(Visited 2,303 times, 1 visits today)
  • REB

    The primary outcome of this study was clinical failure rate, defined as “worsening of symptoms or of otoscopic signs of infection (primarily tympanic-membrane bulging)” or lack of “complete or nearly complete resolution of symptoms and signs attributable to acute otitis media by the end of treatment.” The severity of symptom score, the data you extracted and displayed graphically here, was a secondary outcome. Clinical trials are designed and registered long before journal editors see the manuscripts. They are unable to change the way the trial was designed. The editors graphed the primary outcome in the video summary of the article, which can be found here: http://www.nejm.org/doi/full/10.1056/NEJMoa1606043#t=article.

  • Paul Glasziou

    I have two problems with video abstract abstract. First, the graph in the video abstract does not plot the data over time. Second, as a parent and GP, I am much more concerned with the child’s symptoms than the appearance of the ear drum which is why I was interested to plot the symptom score – which the video fails to describe. Wouldn’t it have helped the reader/user though to plot both the primary and secondary endpoints in the body of the paper?