Every week this new series will uncover an unreported clinical trial
Ben Goldacre, Nicholas DeVito, Carl Heneghan, Síle Lane, Richard Stephens
Clinical trials are the gold standard in medicine. But trial results are still routinely left unreported. [1,2] This problem has been well documented since at least 1986, and has a negative impact on patient care: we cannot make informed choices about treatments when the results of clinical trials are still being withheld from doctors, researchers, and patients. [3,4]
Here we describe a new regular initiative with The BMJ. Every week, we will publish a brief piece describing one important unreported trial that could be used to improve patient care. We hope that this will make the rather abstract issue of publication bias more concrete; shed light on the problem; provide a forum to discuss challenges and misunderstandings around trial reporting; and, ultimately, improve reporting rates.
Our initial sample of unreported trials will be drawn from those recently breaching the FDA Amendments Act of 2007 (FDAAA). This law requires that certain clinical trials in the US report results within 12 months of completion, and was initially hailed as proof that publication bias has now been addressed. Sadly, audits estimate that only one trial in five has complied. [5,6] Furthermore, while the FDA can levy fines at $10 000 per day for every overdue trial, no such fine has ever been levied. [7]
The “Final Rule” that clarified details around the implementation of FDAAA was published in 2016. This gave much needed clarity on FDAAA’s numerous complex loopholes: the global ethical obligation is that all trials must report results, but FDAAA still permits various categories of trial to be left undisclosed. Nonetheless, we have reviewed this law in detail, and we have implemented all its loopholes in software form, as described in our preprint paper. [10]
Every day, our code automatically downloads the whole of ClinicalTrials.gov. Every day, we identify every individual new unreported trial—the moment it goes overdue. All this data is searchable, and ranked, live online at our interactive website: FDAAA.trialstracker.net.
We think our live TrialsTracker approach is superior to a static estimate of overall reporting rates in a journal paper. The data on our TrialsTracker updates daily: any sponsor who improves is rewarded, immediately, and we hope that this will help incentivise improvement. Our tool is also searchable:
we don’t just give overall performance statistics for sponsors; we make it easy to find the individual unreported trials. This is by design. We believe that individual accountability is an important motivator for change. [11]
But more than this, we have made our tool available to help those who want to comply with their obligations: we don’t just publish an overall ranking; we want to help institutions and companies find unreported trials, and address them. This is more than a theoretical aspiration. We already have positive feedback from staff at US universities who have used our tool to find and report unreported trials that had previously been overlooked. [12]
That is the context for this series. Every week, we will cover one individual unreported trial. We will describe the interventions, the population, the number of participants, and the outcomes measured. We will briefly discuss its clinical relevance. We will identify the sponsor, and principal investigators, responsible for the trial. These pieces will be written in collaboration with some of the thousands of patients and patient organisations who are part of the AllTrials campaign. We welcome offers of wider collaboration from patients, and suggestions for individual unreported trials to discuss.
Trials are required by law under the FDAAA to report their results directly onto ClinicalTrials.gov in the form of structured tables. [13,14] This registry carries numerous advantages over journal publication: registry reports have been shown to be more complete, especially on adverse events, while journals have a track record of permitting outcome misreporting and other shortcomings. [15,16,17]
Crucially, registry reports also bypass the long editorial delays at journals. Uploading results to ClinicalTrials.gov is our criteria for a trial to be reported, because this is what the law—the FDAAA—requires.
It is over four decades since non-reporting of clinical trials was first reported. The AllTrials campaign is now supported by 90 000 individuals and over 700 organisations including patient groups, professional bodies, funders, and government bodies such as NICE and IQWiG. [18,19]
It is time for this problem to be fixed. We look forward to positive and productive discussions in the responses to this series; we hope that these discussions will be focused on getting all trials reported.
Ben Goldacre is a doctor, author, and director of the EBM DataLab at the University of Oxford. He co-founded the AllTrials campaign for trials transparency.
Competing interests: BG has received research funding from the Laura and John Arnold Foundation, the Wellcome Trust, the Oxford Biomedical Research Centre, the NHS National Institute for Health Research School of Primary Care Research, the Health Foundation, and the World Health Organization; he also receives personal income from speaking and writing for lay audiences on the misuse of science. ND is employed on BG’s LJAF grant.
Nicholas J DeVito is a researcher at the EBM Datalab at the University of Oxford.
Competing interests: ND is employed on BG’s LJAF grant.
Carl Heneghan is professor of EBM at the University of Oxford, director of CEBM, and a GP. He is also a founder of the AllTrials campaign.
Competing interests: CH jointly runs the Evidence Live conference with The BMJ and is a founder of the AllTrials campaign. He has received expenses and payments for his media work. He has received expenses from the World Health Organization (WHO) and the US FDA, and holds grant funding from the NIHR, the National School of Primary Care Research, the Wellcome Trust, and the WHO. He has published previously on IVF “Add-on treatments” in The BMJ.
Síle Lane is head of international campaigns and policy at Sense about Science, an independent charity that challenges the misrepresentation of evidence in public life. Síle runs AllTrials, the global campaign for clinical trial transparency.
Competing interests: SL is employed by Sense about Science in part to run the AllTrials campaign for clinical trials transparency.
Richard Stephens is a survivor of two cancers and a heart emergency, and has had multiple morbidities and late effects. He chairs the NCRI’s Consumer Forum and sits on NHS England’s Cancer Patient Experience Advisory Group. He is a founder member of the AllTrials and useMYdata movements. He is joint editor in chief of BMC’s Journal of Research Involvement and Engagement.
References:
1] Song F, Parekh S, Hooper L, et al. Dissemination and publication of research findings: an updated review of related biases. Health Technol Assess 2010;14:iii, ix – xi, 1–193.
2] Schmucker C, Schell LK, Portalupi S, et al. Extent of non-publication in cohorts of studies approved by research ethics committees or included in trial registries. PLoS One 2014;9:e114023.
3] Simes RJ. Publication bias: the case for an international registry of clinical trials. J Clin Oncol 1986;4:1529–41.
4] AllTrials.net. AllTrials. http://www.alltrials.net/ (accessed 14 Feb 2018).
5] Anderson ML, Chiswell K, Peterson ED, et al. Compliance with results reporting at ClinicalTrials.gov. N Engl J Med 2015;372:1031–9.
6] Prayle AP, Hurley MN, Smyth AR. Compliance with mandatory reporting of clinical trial results on ClinicalTrials. gov: cross sectional study. The BMJ 2012;344:d7373.
7] Piller C, Bronshthein T. Faced with public pressure, research institutions step up reporting of clinical trial results. STAT. 2018.https://www.statnews.com/2018/01/09/clinical-trials-reporting-nih/
(accessed 14 Mar 2018).
8] World Medical Association. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA 2013;310:2191–4.
9] World Health Organization. WHO Statement on Public Disclosure of Clinical Trial Results. 2015.http://www.who.int/ictrp/results/WHO_Statement_results_reporting_clinical_trials.pdf
10] DeVito NJ, Bacon S, Goldacre B. FDAAA TrialsTracker: A live informatics tool to monitor compliance with FDA requirements to report clinical trial results. bioRxiv. 2018;:266452. doi:10.1101/266452
11] Goldacre B. How to get all trials reported: audit, better data, and individual accountability.
PLoS Med 2015;12:e1001821.
12] Goldacre B. Our FDAAA TrialsTracker is already helping to get new trials reported! EBMdatalab.net. 2018. https://ebmdatalab.net/our-fdaaa-trialstracker-is-already-helping-to-get-new-trials-reported/
(accessed 14 Mar 2018).
13] Zarin DA, Tse T, Williams RJ, et al. The ClinicalTrials.gov results database–update and key issues. N
Engl J Med 2011;364:852–60.
14] Zarin DA, Tse T, Williams RJ, et al. Trial Reporting in ClinicalTrials.gov – The Final Rule. N Engl J Med 2016;375:1998–2004.
15] Riveros C, Dechartres A, Perrodeau E, et al. Timing and completeness of trial results posted at ClinicalTrials.gov and published in journals. PLoS Med 2013;10:e1001566; discussion e1001566.
16] Jones CW, Keil LG, Holland WC, et al. Comparison of registered and published outcomes in randomized controlled trials: a systematic review. BMC Med 2015;13:282.
17] Turner L, Shamseer L, Altman DG, et al. Does use of the CONSORT Statement impact the completeness of reporting of randomised controlled trials published in medical journals? A Cochrane review. Syst Rev 2012;1:60.
18] AllTrials Campaign. All trials registered. All results reported. 2013.http://www.alltrials.net/wp-content/uploads/2013/09/What-does-all-trials-registered-and-reported-mean.pdf
19] Supporters – Organisations. AllTrials.net. http://www.alltrials.net/supporters/organisations/
(accessed 14 Mar 2018).