The FDA should demand adequate, controlled studies with long term follow up, and make data publicly available, before granting full approval to covid-19 vaccines, says Peter Doshi<\/p>\n
<\/p>\n
On 28 July 2021, Pfizer and BioNTech <\/span>posted updated results<\/span><\/a> for their ongoing phase 3 covid-19 vaccine trial. The preprint came almost a year to the day after the historical trial commenced, and nearly four months since the companies announced <\/span>vaccine efficacy estimates \u201cup to six months.\u201d<\/span><\/a><\/p>\n But you won\u2019t find 10 month follow-up data here. While the preprint is new, the results it contains aren\u2019t particularly up to date. In fact, the paper is based on the same data cut-off date (13 March 2021) as the <\/span>1 April press release<\/span><\/a>, and its topline efficacy result is identical: 91.3% (95% CI 89.0 to 93.2) vaccine efficacy against symptomatic covid-19 through \u201cup to six months of follow-up.\u201d<\/span><\/p>\n The 20 page preprint matters because it represents the most detailed public account of <\/span>the pivotal trial data Pfizer submitted<\/span><\/a> in pursuit of the world\u2019s first \u201cfull approval\u201d of a coronavirus vaccine from the Food and Drug Administration. It deserves careful scrutiny.<\/span><\/p>\n The elephant named \u201cwaning immunity\u201d<\/b><\/p>\n Since late last year, we\u2019ve heard that Pfizer and Moderna\u2019s vaccines are \u201c95% effective\u201d with even greater efficacy against severe disease (<\/span>\u201c100% effective,\u201d<\/span><\/a> Moderna said).<\/span><\/p>\n Whatever one thinks about the \u201c95% effective\u201d claims (my thoughts are <\/span>here<\/span><\/a>), even the most enthusiastic commentators have acknowledged that measuring vaccine efficacy two months after dosing says little about just how long vaccine-induced immunity will last. \u201cWe\u2019re going to be looking very intently at the durability of protection,\u201d <\/span>Pfizer senior vice president William Gruber<\/span><\/a>, an author on the <\/span>recent preprint<\/span><\/a>, told the FDA\u2019s advisory committee last December.<\/span><\/p>\n The concern, of course, was decreased efficacy over time. \u201cWaning immunity\u201d is a <\/span>known problem for influenza vaccines<\/span><\/a>, with some studies showing near zero effectiveness after just three months, meaning a vaccine taken early may ultimately provide no protection by the time \u201cflu season\u201d arrives some months later. If vaccine efficacy wanes over time, the crucial question becomes what level of effectiveness will the vaccine provide when a person is actually exposed to the virus? Unlike covid vaccines, <\/span>influenza vaccine performance<\/span><\/a> has always been judged over a full season, not a couple months.<\/span><\/p>\n And so the recent reports from Israel\u2019s Ministry of Health caught my eye. In <\/span>early July,<\/span><\/a> they reported that efficacy against infection and symptomatic disease \u201cfell to 64%.\u201d By late July it had fallen to <\/span>39%<\/span><\/a> where Delta is the dominant strain. This is very low. For context, the <\/span>FDA\u2019s expectation<\/span><\/a> is of \u201cat least 50%\u201d efficacy for any approvable vaccine.<\/span><\/p>\n Now Israel, which almost exclusively used Pfizer vaccine, has begun administering a third \u201cbooster\u201d dose to <\/span>all adults over 40<\/span><\/a>. And starting 20 September 2021, the US plans to follow suit for <\/span>all \u201cfully vaccinated\u201d adults<\/span><\/a> eight months past their second dose.<\/span><\/p>\n Delta may not be responsible<\/b><\/p>\n Enter Pfizer\u2019s preprint. As an RCT reporting \u201cup to six months of follow-up,\u201d it is notable that evidence of waning immunity was already visible in the data by the 13 March 2021 data cut-off.\u00a0\u00a0<\/span><\/p>\n \u201cFrom its peak post-dose 2,\u201d the <\/span>study authors write<\/span><\/a>, \u201cobserved VE [vaccine efficacy] declined.\u201d From 96% to 90% (from two months to <4 months), then to 84% (95% CI 75 to 90) \u201cfrom four months to the data cut-off,\u201d which, by my calculation (see footnote at the end of the piece), was about one month later.<\/span><\/p>\n But although this additional information was available to Pfizer in April, it was not published until the end of July.<\/span><\/p>\n And it\u2019s hard to imagine how the Delta variant could play a real role here, for <\/span>77% of trial participants<\/span><\/a> were from the United States, where <\/span>Delta was not established<\/span><\/a> until months after data cut-off.<\/span><\/p>\n Waning efficacy has the potential to be far more than a minor inconvenience; it can dramatically change the risk-benefit calculus. And whatever its cause\u2014intrinsic properties of the vaccine, the circulation of new variants, some combination of the two, or something else\u2014the bottom line is that vaccines need to be effective.<\/span><\/p>\n Until new clinical trials demonstrate that boosters increase efficacy above 50%, without increasing serious adverse events, it is unclear whether the 2-dose series would even meet the FDA\u2019s approval standard at six or nine months.\u00a0\u00a0<\/span><\/p>\n The \u201csix month\u201d preprint based on the 7% of trial participants who remained blinded at six months<\/b><\/p>\n The final efficacy timepoint reported in Pfizer\u2019s preprint is \u201cfrom four months to the data cut-off.\u201d The confidence interval here is wider than earlier time points because only half of trial participants (53%) made it to the four month mark, and mean follow-up is around 4.4 months (see footnote).<\/span><\/p>\n This all happened because <\/span>starting last December<\/span><\/a>, Pfizer allowed all trial participants to be formally unblinded, and placebo recipients to get vaccinated. By 13 March 2021 (data cut-off), 93% of trial participants (41,128 of 44,060; <\/span>Fig 1<\/span><\/a>) were unblinded, officially entering \u201copen-label followup.\u201d (Ditto for Moderna: by mid April, <\/span>98% of placebo recipients had been vaccinated<\/span><\/a>.)<\/span><\/p>\n Despite the reference to \u201csix month safety and efficacy\u201d in the preprint\u2019s title, the paper only reports on vaccine efficacy \u201cup to six months,\u201d but <\/span>not <\/span>from<\/span><\/i> six months<\/span><\/a>. This is not semantics, as it turns out only 7% of trial participants actually reached six months of blinded follow-up (\u201c8% of BNT162b2 recipients and 6% of placebo recipients had \u22656 months follow-up post-dose 2.\u201d) So despite this preprint appearing a year after the trial began, it provides no data on vaccine efficacy past six months, which is the period Israel says vaccine efficacy has dropped to 39%.<\/span><\/p>\n It is hard to imagine that the <10% of trial participants who remained blinded at six months (which presumably further dwindled after 13 March 2021) could constitute a reliable or valid sample to produce further findings. And the preprint does not report any demographic comparisons to justify future analyses.<\/span><\/p>\n Severe disease<\/b><\/p>\n With the US awash in news about rising cases of the Delta variant, including among the \u201cfully vaccinated,\u201d the vaccine\u2019s efficacy profile is in question. But some medical commentators are delivering an upbeat message. Former FDA commissioner Scott Gottlieb, who is on Pfizer\u2019s board, <\/span>said<\/span><\/a>: \u201cRemember, the original premise behind these vaccines were [sic] that they would substantially reduce the risk of death and severe disease and hospitalization. And that was the data that came out of the initial clinical trials.\u201d<\/span><\/p>\n Yet, the trials were <\/span>not designed to study severe disease<\/span><\/a>. In the data that supported Pfizer\u2019s EUA, <\/span>the company itself<\/span><\/a> characterized the \u201csevere covid-19\u201d endpoint results as \u201cpreliminary evidence.\u201d Hospital admission numbers were not reported, and <\/span>zero covid-19 deaths<\/span><\/a> occurred.<\/span><\/p>\n In the preprint, high efficacy against \u201csevere covid-19\u201d is reported based on all follow-up time (one event in the vaccinated group vs 30 in placebo), but the number of hospital admissions is not reported so we don\u2019t know which, if any, of these patients were ill enough to require hospital treatment. (In Moderna\u2019s trial, data last year showed that 21 of 30 \u201csevere covid-19\u201d cases were not admitted to hospital; <\/span>Table S14<\/span><\/a>).<\/span><\/p>\n And on preventing death from covid-19, there are too few data to draw conclusions\u2014a total of <\/span>three covid-19 related deaths<\/span><\/a> (one on vaccine, two on placebo). There were 29 total deaths during blinded follow-up (15 in the vaccine arm; 14 in placebo).<\/span><\/p>\n The crucial question, however, is whether the waning efficacy seen in the primary endpoint data also applies to the vaccine\u2019s efficacy against severe disease. Unfortunately, Pfizer\u2019s new preprint does not report the results in a way that allows for evaluating this question.<\/span><\/p>\n Approval imminent without data transparency, or even an advisory committee meeting?<\/b><\/p>\n Last December, with limited data, the FDA granted Pfizer\u2019s vaccine an EUA, enabling access to all Americans who wanted one. It sent a clear message that the FDA could both address the enormous demand for vaccines without compromising on the science. A \u201cfull approval\u201d could remain a high bar.<\/span><\/p>\n But here we are, with FDA reportedly <\/span>on the verge of granting a marketing license<\/span><\/a> 13 months into the <\/span>still ongoing, two year pivotal trial<\/span><\/a>, with no reported data past 13 March 2021, unclear efficacy after six months due to unblinding, evidence of waning protection irrespective of the Delta variant, and limited reporting of safety data. (The preprint reports \u201cdecreased appetite, lethargy, asthenia, malaise, night sweats, and hyperhidrosis were new adverse events attributable to BNT162b2 not previously identified in earlier reports,\u201d but provides no data tables showing the frequency of these, or other, adverse events.)<\/span><\/p>\n It\u2019s not helping matters that <\/span>FDA now says it won\u2019t convene its advisory committee <\/span><\/a>to discuss the data ahead of approving Pfizer\u2019s vaccine. (Last August, to address vaccine hesitancy, the agency had \u201c<\/span>committed to use an advisory committee<\/span><\/a> composed of independent experts to ensure deliberations about authorization or licensure are transparent for the public.\u201d)<\/span><\/p>\n Prior to the preprint, my view, along with a group of around 30 clinicians, scientists, and patient advocates, was that there were simply <\/span>too many open questions<\/span><\/a> about all covid-19 vaccines to support approving any this year. The preprint has, unfortunately, addressed very few of those open questions, and has raised some new ones.<\/span><\/p>\n I reiterate <\/span>our call<\/span><\/a>: \u201cslow down and get the science right\u2014there is no legitimate reason to hurry to grant a license to a coronavirus vaccine.\u201d<\/span><\/p>\n FDA should be demanding that the companies complete the two year follow-up, as originally planned (even without a placebo group, much can still be learned about safety). They should demand adequate, controlled studies using patient outcomes in the now substantial population of people who have recovered from covid. And regulators should bolster public trust by helping ensure that everyone can <\/span>access the underlying data<\/span><\/a>.<\/span><\/p>\n Peter Doshi<\/strong>, senior editor, <\/span><\/em>The BMJ.<\/span><\/p>\n Competing interests: <\/b>I helped organize the Coalition Advocating for Adequately Licensed Medicines (CAALM), which has <\/span>formally petitioned the FDA<\/span><\/a> to refrain from fully approving any covid-19 vaccine this year (docket <\/span>FDA-2021-P-0786<\/span><\/a>). A full list of competing interests is available <\/span>here<\/span><\/a>.\u00a0The views and opinions expressed here are mine and do not necessarily reflect official policy or the position of the University of Maryland.<\/span><\/em><\/p>\n