The merits of publicly releasing study protocols and statistical analysis plans have been overlooked in the vigorous debate about releasing individual participant level data. [1] In this Opinion, we argue that clinical study reports, which contain these methodological documents and are submitted to regulators, should be made available to help clinicians and patients better make informed decisions. [2] We demonstrate this using the example of doxylamine-pyridoxine.

Nausea and vomiting in women in their first trimester is common, and it can severely impair function, cause misery for a substantial portion of pregnancy, and even necessitate hospitalization. So it is no surprise that effective treatments have been sought for decades. Doxylamine-pyridoxine has a complex regulatory history. Trials of doxylamine-pyridoxone reported in 1959 and 1971 had opposing findings. [3] A 1974 trial lost 31% of participants to follow-up and was marred by questions about data integrity; its final results are now available. [3] In 1983 the medication was voluntarily removed from the market in the midst of eventually dismissed litigation related to birth defects. [4]

The best study to address the efficacy of doxylamine-pyridoxine was a 2009 placebo-controlled trial in women with nausea and vomiting during pregnancy which found a statistically significant decrease in symptom scores compared to placebo. It was on this basis that the FDA approved doxylamine-pyridoxine in 2013 for nausea and vomiting during pregnancy, and was heralded by FDA reviewers as a triumph of rigorous assessments over past “decisions that are not science-based”. [5]

However, this was not the end of the story. After a years-long struggle to obtain the clinical study report from Health Canada, a re-analysis one of us conducted found that the medication was in fact not effective.[4, 6, 7, 8] The 2009 study had found the difference in symptom improvements between groups was less than one point on a 13-point scale; the upper limit of the confidence interval, 1.25 points, was below the prespecified “intended clinical benefit” of 3 points and there are no validation studies suggesting an improvement of this magnitude is clinically important. [6-8] However, two reports of the trial published by the investigators from 2010 and 2016 failed to report the pre-specified 3-point “intended clinical benefit” and the detailed FDA review did not reconcile the findings with what was pre-specified. [6,7] The pre-specified 3 point difference came to light only after the release of the clinical study report, which included details of the sample size calculation that did not make it into the journal publications.

In addition, although hyperemesis gravidarum was pre-specified as a secondary outcome in the clinical trial, it was not reported in either the original 2010 report or the 2016 report that included some previously unpublished results. [6-8] The clinical study report showed that only one trial participant had a hospital visit with hyperemesis gravidarum, and that individual happened to be in the doxylamine-pyridoxine group. [8] Yet the FDA speculated about the number of hospitalizations that may have been caused by the medication’s removal from the market in 1983. [5] This conjecture was based on an observational study that is not supported by the primary result of the clinical trial. Again, this finding was only found by a re-analysis of the clinical study report.

Details such as the ones contained in clinical study reports, but not usually in journal articles are important. Ideally, all important details of trials would be available in published reports, but we know that reporting guidelines are often not followed. We argue that the FDA should make available the clinical study reports after medications are approved, similar to what the EMA does and as Health Canada has announced it will do. [2,9]

The EMA explicitly recognizes that disclosing the detailed clinical study report can lead to better decisions, as others may notice something missed during the regulatory review. No regulatory process is infallible. Despite the careful review conducted and publicly posted by the FDA that included several additional statistical analyses, the fact that the observed difference was short of the pre-specified “intended clinical effect” was missed. If the FDA had posted the clinical study report when it approved doxylamine-pyridoxine, this may have been identified before the multiple reports, before the guideline recommendations, and before American women started using this medication.

There are reasons that clinical study reports are not released today and some of them are valid. Investigators may not want clinical study reports made publicly available until after they have had an opportunity to publish any secondary analyses. The solution to this problem is theoretically simple: allow trial sponsors to delay regulatory approval (and therefore clinical study report release) until all important analyses are finished. Since the purpose of the secondary analyses is presumably to answer important questions about the treatment, why approve it before all the important work is finished and vetted? Also, although clinical study reports contain individual participant level data, the reports with the methodological details can be released without the data if needed. In the case of doxylamine-pyrodoxine, the methodological details such as the sample size calculation would have been sufficient to call the conclusion of original report into question. Concerns about releasing individual participant level data do not apply to the methodological details in the clinical study report.

Increased transparency could actually make it more difficult to interpret trial findings if there are multiple conflicting reports. It is difficult to guarantee the quality of subsequent reports and the process of re-analyzing trial findings can introduce misunderstanding and errors. However, there is no definitive way to ensure that trials are conducted and reported properly the first time, but that does not stop them from being done. So why would we draw the line here?

Clinical study reports may contain proprietary information that trial sponsors do not want shared with competitors for legitimate reasons. This could include details of a proprietary assay or technique. Under these extraordinary circumstances, details about the proprietary assay or technique could be redacted before release. These situations should be rare since clinical study reports usually report the clinical effects of a treatment.

If the purpose of clinical research is to improve care and promote health—and not just to get products approved—that purpose should be prioritized ahead of other considerations. The doxylamine-pyridoxine trial was reported three times and it was approved twice by the FDA, but the details in the clinical study report indicate that the medication effect is not clinically significant. [6,8] While there are some downsides to publicly releasing clinical study reports, transparency can help regulators, medical journals, and clinicians improve the care of the patients we serve.

Nav Persaud, scientist, Centre for Urban Health Solutions, St. Michael’s Hospital, Toronto, ON, and Assistant Professor, Department of Family and Community Medicine, University of Toronto Faculty of Medicine, Toronto, ON

 

 

 

Art Slutsky, vice president research, St. Michael’s Hospital, Toronto, ON, and Professor, University of Toronto, ON

Acknowledgement: The authors thank Andreas Laupacis for helpful comments.

Competing interests: There are no competing interests to report.

 

 

References:

1. 1. Hoffmann T, Glasziou P, Beller E, Goldacre B, Chalmers I. Focus on sharing individual patient data distracts from other ways of improving trial transparency.
      BMJ. 2017;357:j2782.
   2. Persaud N, Doshi P. North American regulatory agencies can and should make clinical trial data publicly available. CMAJ 2016 Feb 2;188(2):96-7. doi: 10.1503/cmaj.150679.
   3. Zhang R, Persaud N. 8-Way Randomized Controlled Trial of Doxylamine, Pyridoxine and Dicyclomine for Nausea and Vomiting during Pregnancy: Restoration of Unpublished Information. PLOS ONE 2017;12(1):e0167609. doi: 10.1371/journal.pone.0167609
   4. U.S FDA. Drug approval package – Diclegis (doxylamine succinate and pyridoxine hydrochloride) delayed-release tablets 2014 [updated March 25, 2014. Available from: https://www.accessdata.fda.gov/drugsatfda_docs/nda/2013/021876Orig1s000TOC.cfm accessed Nov 14 2017.
   5. Slaughter SR, Hearns-Stokes R, van der Vlugt T, et al. FDA approval of doxylamine-pyridoxine therapy for use in pregnancy. The New England Journal of Medicine 2014;370(12):1081-3. doi: 10.1056/NEJMp1316042 [published Online First: 2014/03/22]
   6. Koren G, Clark S, Hankins GD, et al. Effectiveness of delayed-release doxylamine and pyridoxine for nausea and vomiting of pregnancy: a randomized placebo controlled trial. American Journal of Obstetrics & Gynecology 2010;203(6):571.e1-7. doi: 10.1016/j.ajog.2010.07.030 [published Online First: 2010/09/17]
   7. Koren G, Hankins GDV, Clark S, et al. Effectiveness of doxylamine-pyridoxine for morning sickness. American Journal of Obstetrics & Gynecology 2016214(5):664-66. doi: 10.1016/j.ajog.2016.01.186
   8. Persaud N, Meaney C, El-Emam K, et al. Doxylamaine-pyridoxine for nausea and vomiting of pregnancy randomized placebo control trial: prespecified analyses and reanalysis PLOS ONE 2018 http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0189978
   9. European Medicines Agency. Clinical data – online access to clinical data for medicinal products for human use 2017 [updated December 09, 2017; cited 2018 January 10]. Available from: https://clinicaldata.ema.europa.eu/web/cdp/home accessed January 10 2018.