Joshua D. Wallach, Harlan M. Krumholz, Joseph S. Ross

Over the past few years, the clinical trial data sharing landscape has changed dramatically. As a result of concerns regarding the frequency of unpublished and selectively reported clinical trial results, various stakeholders, including pharmaceutical companies, began to embrace open science practices. For instance, in 2013, GlaxoSmithKline (GSK) launched ClinicalStudyDataRequest.com, a platform that allows independent researchers to request access to the raw data from approximately 1,500 studies (i.e., individual patient level data) conducted by thirteen pharmaceutical companies and a number of academic research funders. Other industry leaders, such as Johnson & Johnson and Bristol-Myer Squibb, have partnered with academic institutions to facilitate access to their trial data (Yale University Open Data Access (YODA) project and Supporting Open Access for Researchers (SOAR) initiative, respectively).

Now that the wall surrounding clinical trial data has started to come down, unique opportunities have emerged for researchers to test new hypothesis, synthesize evidence, and re-analyze study results. In particular, individual patient level data can play a critical role when evaluating drug and device safety. Previous studies have consistently shown that adverse events are often “neglected, restricted, distorted, and silenced” in publicly available summary-level data sources, such as trial publications and registries used for results reporting (e.g., ClinicalTrials.gov). Unlike publicly available data sources, which often report composite outcomes and rarely present information about all safety events, individual patient level data can be used to consistently identify and classify potentially missing or poorly reported outcomes, thereby reducing the impact of selective adverse event reporting in summary level data sources.

This week in The BMJ, we published our study assessing the influence of individual patient level data, made available through ClinicalStudyDataRequest.com, on the conclusions drawn from safety-related meta-analyses examining relatively rare adverse events. Our evaluation focused on rosiglitazone, a medication used for Type 2 Diabetes Mellitus manufactured by GSK under the brand name Avandia. Rosiglitazone was approved by the US Food and Drug Administration (FDA) and European Medicines Agency (EMA) in 1999 and 2000, respectively, despite initial concerns about the risk of heart failure.

However, in 2007, when a meta-analysis was published suggesting a 43% increased risk of myocardial infarction, there were widespread concerns surrounding the cardiovascular safety of rosiglitazone. Since that time, there have been numerous regulatory evaluations and multiple other analyses, using various approaches and publicly available summary level data sources, that have reported conflicting findings related to rosiglitazone’s cardiovascular risk. Although rosiglitazone has been removed from the market in most countries, and use has rapidly dropped, rosiglitazone is still available in the US. 

Using individual patient level data available for 33 GSK trials, our analyses offered several insights. First, we identified a greater number of myocardial infarctions and fewer cardiovascular deaths reported in the individual patient level data compared to what had been reported for the same trials in previous reviews utilizing only publicly available summary level data. Second, we confirmed previous reports of an increased risk of heart failure—an approximately 50% increased risk. Third, we found that rosiglitazone was associated with an increased risk of myocardial infarction risk—an approximately 17% increased risk. Both of these inferences held up regardless of how we analyzed the data, as we tested multiple different analytic approaches (fixed or random-effects, odds ratios or relative risks, and including or excluding trials with zero adverse events in all intervention arms).   

What made our analyses particularly interesting was that after we analyzed individual patient level data for these 33 trials, we combined these data with publicly available summary level data from over 100 other clinical trials. In these analyses, we found that the magnitude of the risk of myocardial infarction decreased and was no longer statistically significant by conventional standards. While there are many potential reasons for these observed changes, we suspect that these differences are a result of combining the individual patient level data with lower quality data from the studies without individual patient level data, which often had smaller sample sizes and less rigorous designs. 

Our experience using shared individual patient level data suggests that meta-analyses focused on drug safety need raw data for accurate classification and synthesis. ClinicalStudyDataRequest.com allowed us to update insights regarding the safety of a drug approved over two decades ago, taking advantage of dozens of clinical trials that had been conducted over the years. With data from thousands of other trials now available across different platforms, the research community should embrace these resources to update and improve our understanding of drug and device safety to inform patients and clinicians. 

Joshua D. Wallach is an Assistant Professor of Epidemiology at the Yale School of Public Health. Twitter: @JoshuaDWallach

Harlan M. Krumholz is the Harold H. Hines, Jr. Professor of Medicine and Public Health at Yale University. Twitter: @hmkyale

Joseph S. Ross is Professor of Medicine and of Public Health at Yale University. Twitter: @jsross119

Competing Interests: Joshua D. Wallach has received research support through the Meta Research Innovation Center at Stanford (METRICS) and the Collaboration for Research Integrity and Transparency at Yale, from the Laura and John Arnold Foundation. He receives research support through the Yale-Male Clinic Center for Excellence in Regulatory Science and Innovation (CERSI) (U01FD005938). Harlan Krumholz received research support through Yale from Johnson and Johnson to develop methods of clinical trial data sharing, from Medtronic, Inc. and the Food and Drug Administration (FDA) to develop methods for postmarket surveillance of medical devices (U01FD004585), from the Centers of Medicare and Medicaid Services (CMS) to develop and maintain performance measures that are used for public reporting, received payment from the Arnold & Porter Law Firm for work related to the Sanofi clopidogrel litigation and from the Ben C. Martin Law Firm for work related to the Cook IVC filter litigation, chairs a Cardiac Scientific Advisory Board for UnitedHealth, is a participant/participant representative of the IBM Watson Health Life Sciences Board, is a member of the Advisory Board for Element Science and the Physician Advisory Board for Aetna, and is the founder of Hugo, a personal health information platform. Joseph S. Ross received research support through Yale from Johnson and Johnson to develop methods of clinical trial data sharing, from Medtronic, Inc. and the Food and Drug Administration (FDA) to develop methods for postmarket surveillance of medical devices (U01FD004585), from the Centers of Medicare and Medicaid Services (CMS) to develop and maintain performance measures that are used for public reporting, from the FDA to establish a Center for Excellence in Regulatory Science and Innovation (CERSI) at Yale University and the Mayo Clinic (U01FD005938), from the Blue Cross Blue Shield Association to better understand medical technology evaluation, and from the Agency for Healthcare Research and Quality (R01HS022882).