On 29 April 2020 The Lancet published online the results of a study, conducted in China, reporting that “remdesivir was not associated with statistically significant clinical benefits in the treatment of severe covid-19. However, the numerical reduction in time to clinical improvement in those treated earlier requires confirmation in larger studies.” [2] On the same day, by remarkable coincidence, Gilead Sciences, Inc., reported the preliminary results of the ACTT-1 study of the same drug, which it manufactures, showing that patients who received the drug on average recovered from their covid-19 symptoms four days earlier than those who received the placebo. [3] Like the Chinese study, the Gilead study found that no fewer patients developed serious complications nor did fewer die. 

Although Gilead concluded that Remdesivir “has not yet been demonstrated to be safe or effective for the treatment of covid-19”, the U.S. National Institute of Arthritis and Infectious Disease, both sponsor and terminator of the study, announced on the same day that:

“Hospitalized patients with advanced covid-19 and lung involvement who received remdesivir recovered faster than similar patients who received placebo” an announcement seized upon by the media waiting for any indication that medical science might have an effective approach to treatment of this devastating viral agent. [4] The AP News proclaimed the results “A 1st” and quoted Anthony Fauci of the National Institutes of Health saying that “What it has proven is that a drug can block this virus,” that “This will be the standard of care,” and that any other potential treatments will now have to be tested against or in combination with Remdesivir. [5] 

However, what was not reported in the Gilead Sciences, Inc. announcement nor in the NIH announcement made on the same day was the fact that the registered primary endpoint of this study was changed on the clinicaltrials.gov website on 16 April 2020. [3,4] The original primary outcome, registered on 20 February 2020, was a seven-point clinical hierarchy focused principally on the most severe objective consequences of infection with covid-19, including death. On 16 April 2020, the revised primary endpoint became the “day of recovery”, reflecting only the lowest illness severity status of the originally registered primary endpoint. Because Clinicaltrials.gov is a public website, the change in primary endpoint was disclosed publicly. Nonetheless, clinicaltrials.gov is not a website that is in plain view to most of the public. Given the widespread dissemination of the announcement of the results themselves by Gilead Sciences, Inc. and by the NIH, the lack of mention of the changed primary endpoint appears an oversight in communication. 

Changing endpoints in major clinical trials is generally frowned upon because the study design was not drafted to focus on the secondary endpoints. Furthermore, while the phenomenon of changing endpoints has been long-recognized, there are data that such changes can be unreliable and lead to overestimates of the intervention effect size. [6-8]     

Moreover, changing endpoints should be done only for good cause since “…failure to prespecify endpoints can introduce bias into a trial and creates opportunities for manipulation.” [9] “The principle (sic) consideration when evaluating whether to modify an endpoint is whether the decision is independent of the data obtained from the trial to date.” [9] 

In this respect, the sponsors were lucky to elevate the one secondary endpoint, out of  the 24 original secondary endpoints, that was already known to be the only promising end-point in the Chinese study, although not statistically significant in China. Nevertheless, the value of recovering four days earlier is debatable, if there is no effect on the occurrence of serious complications, including death. [10] This value, if any, must be compared to the price of the drug, its side effects, and the consequences of false hopes engendered by optimistic projections of as yet unproven benefit. Opinions aside, two days later the FDA  issued an emergency use authorization for remdesivir on the basis of “promising results.” [11] Health and Human Services Secretary, Akex Azar, touted this as “another example of the Trump Administration moving as quickly as possible to use science to save lives,” and a “seamless cooperation between government and private industry under the President’s all-of-America approach to covid-19….” a cooperation no doubt encouraged by the $2.45M in lobbying funds spent by Gilead Sciences during the first quarter of 2020. [12] Shortly after, on 11 May the European Medicines Agency (EMA) also expanded its recommendation for compassionate use of remdesivir to include patients not on mechanical ventilation. [13] 

A preliminary report of the ACTT-1 study was published 22 May in the New England Journal of Medicine, but it did not reveal significant new outcomes data, apart from the fact that the drug apparently failed to show efficacy in Black or Asian populations, or in people from Hispanic or Latino ethnic groups. [14] These findings were not mentioned in the original media releases. [3,4] Since the full study results have not yet been reported, it remains difficult to decide whether the risks of the decision to reprioritize endpoints and terminate the study outweigh the benefits of not doing so. In fact, it now appears unlikely that we will ever be able to weigh properly the true risk/benefit ratio. Since the serious, hard clinical endpoints are now secondary ones, whether Remdesivir improves them or not will carry less “scientific” weight in the strictly evidence-based evaluation of their significance. Additionally, as others have commented, the presence of a reported beneficial primary endpoint will complicate further placebo controlled trials to help settle the outstanding issues. [10] The decision to both change the endpoints and terminate the study prematurely, has left open a variety of critical questions that would have been answered had the study carried on to its conclusion using its original primary and secondary endpoints. As Ethan Weiss at the University of California, San Francisco, summed up the situation, “We’ve squandered an incredible opportunity to do good science”. [10]

Dennis M Bier, Department of Pediatrics, Children’s Nutrition Research Center; Baylor College of Medicine, Houston, Texas, USA.

Arne Astrup, Department of Nutrition, Exercise and Sports, University of Copenhagen.

Conflicts of interest:

In the last 3 years, Dr. Bier has received consultant and/or lecture fees and/or reimbursements for travel, hotel and other expenses from the International Life Sciences Institute; the International Council on Amino Acid Science; Nutrition and Growth Solutions, Inc.; Ajinomoto, Co.; the Lorenzini Foundation; the Nutrition Coalition, the CrossFit Foundation; the International Glutamate Technical Committee; Nestlé S.A.; Ferrero SpA; Indiana University; the National Institutes of Health; Mallinckrodt Pharmaceuticals; the Infant Nutrition Council of America; and the Israel Institute.

In the last 3 years Dr. Astrup has received compensation for advisory boards, scientific councils, consultancy and lectures from BioCare Copenhagen, DK, Dutch Beer Institute, NL, Ferrero, IT.; Groupe Éthique et Santé, FR; International Egg Commission, UK; McCain Foods Limited, USA; Novo Nordisk A/S, DK; Novo Nordisk Foundation, DK; Ritual Inc. Los Angeles, USA; Weight Watchers, USA; Gelesis, USA, Saniona, DK, European Milk Forum, EU, The Nutrition Coalition, USA.  

References:

1. Defoe DA. Journal of the Plague Year: Being Observations or Memorials, of the Most Remarkable Occurrences, As Well Publick as Private, Which Happened in London During the Last Great Visitation in 1665. Printed for E. Nutt at the Royal-Exchange; J Roberts in Warwick-Lane; A Dodd Without Temple-Bar; and J. Graves in St. James St. London, 1722.
2. Wang Y, Zhang D, Du G, et al. Remdesivir in adults with severe COVID-19: a randomised, double-blind, placebo-controlled, multicentre trial. Lancet April 29, 2020. https://doi.org/10.1016/S0140-6736(20)31022-9
3. Gilead Sciences Inc. Gilead announces results from Phase 3 trial of investigational antiviral Remdesivir in patients with severe COVID-19. Study demonstrates similar efficacy with 5- and 10-day dosing durations of Remdesivir. gilead.com April 29, 2020. https://www.gilead.com/news-and-press/press-room/press-releases/2020/4/gilead-announces-results-from-phase-3-trial-of-investigational-antiviral-remdesivir-in-patients-with-severe-covid-19
4. National Institute of Allergy and Infectious Diseases. NIH clinical trial shows Remdesivir accelerates recovery from advanced COVID-19. niaid.gov April 29, 2020. https://www.niaid.nih.gov/news-events/nih-clinical-trial-shows-remdesivir-accelerates-recovery-advanced-covid-19
5. Marchione M. A 1st: US study finds Gilead drug works against coronavirus. AP News April 30, 2020. https://apnews.com/b894f3b39f6915dcc3cc402438deaf0b
6. Goldacre B. Make journals report clinical trials properly. Nature 2016;7:530. doi:10.1038/530007a
7. Chan A-W, Hróbjartsson A, Haahr MT, Gotzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials. JAMA 2004;291:2457. doi:10.1001/jama.291.20.2457
8. Chen T, Li C, Qin R, et al. Comparison of clinical trial changes in primary outcome and reported intervention effect size between trial registration and publication. JAMA Network Open 2019;2:e197242. doi:10.1001/jamanetworkopen.2019.7242
9. Evans S. When and how can endpoints be changed after initiation of a randomized clinical trial? PLOS Clin Trials 2007;2:e18. doi: 10.1371/journal.pctr.0020018
10. Herper M. Inside the NIH’s controversial decision to stop the big Remdesivir study.  Statnews.com May 11, 2020. https://www.statnews.com/2020/05/11/inside-the-nihs-controversial-decision-to-stop-its-big-remdesivir-study/
11. Coronavirus (COVID-19) Update: FDA Issues Emergency Use Authorization for Potential COVID-19 Treatment. For Immediate Release:May 01, 2020. https://www.fda.gov/news-events/press-announcements/coronavirus-covid-19-update-fda-issues-emergency-use-authorization-potential-covid-19-treatment
12. Lupkin S. Gilead Lobbying Rose As Interest In COVID-19 Treatment Climbed. Shots Health News from NPR. https://www.npr.org/sections/health-shots/2020/05/02/849149873/gilead-lobbying-rose-as-interest-in-covid-19-treatment-climbed?t=1588603835076
13. EMA recommends expanding remdesivir compassionate use to patients not on mechanical ventilation. 11th May 2020. https://www.ema.europa.eu/en/news/ema-recommends-expanding-remdesivir-compassionate-use-patients-not-mechanical-ventilation
14. Beigel JH, Tomashek KM, Dodd LE et al. Remdesivir for the Treatment of Covid-19 – Preliminary Report. N Engl J Med. 2020 May 22. doi: 10.1056/NEJMoa2007764.