Pfizer and Moderna didn’t design the trials to evaluate long-term efficacy or the more important outcomes of preventing hospitalization, death, or transmission.
The randomized trials did collect adverse event data, including the presence of mild symptoms (such as a fever) and more serious events requiring hospitalization or leading to death. Most vaccines generate some mild adverse reactions in some people, and there were considerably more adverse reactions after the mRNA vaccines compared to the placebo.
That is annoying, but not a major issue. We care about severe health outcomes. The key question is whether the vaccine’s efficacy outweighs the risks of severe adverse reactions.
The Fraiman study uses data from the same Pfizer- and Moderna-sponsored randomized trials presented to the Food and Drug Administration for vaccine approval, but with two innovations that provide additional information.
First, the study pools data from both mRNA vaccines to increase the sample size, which decreases the confidence intervals’ size and the uncertainty about the estimated harms.
Second, the study focuses only on the severe adverse events plausibly caused by the vaccines. Serious adverse events, such as gunshot wounds, suicide, animal bites, foot fractures, and back injury, are unlikely to be caused by a vaccine, and cancer is unlikely to be caused by a vaccine within a few months after vaccination. By removing such random noise, the ability (statistical power) to detect genuine problems increases. If there is no excess risk, shorter confidence intervals bolster confidence in the safety of the vaccines.
Moreover, Fraiman and colleagues blinded the process where they classified the clinical events as AESIs. Adjudicators didn’t know whether the individual had received the vaccine or the placebo. Hence, any criticism of so-called p-hacking is unwarranted.
So, what were the results? There were 139 AESIs among the 33,986 people vaccinated, one for every 244 people. That may sound bad, but those numbers mean nothing without comparison against a control group. There were 97 AESIs among the 33,951 people who received a placebo. Combining these numbers implies 12.5 vaccine-induced AESIs for every 10,000 people vaccinated, with a 95 percent confidence interval of 2.1 to 22.9 per 10,000 people. To phrase it differently, there is one additional AESI for every 800 people vaccinated (95 percent CI: 437-4762).
That is very high for a vaccine. No other vaccine on the market comes close.
The numbers for the Pfizer and Moderna vaccines are 10 and 15 additional events per 10,000 people, respectively, so both vaccines contributed to the finding. The numbers are similar enough that we can’t confidently say that one is safer than the other. Most excess AESIs were coagulation disorders. For the Pfizer vaccine, there was also an excess of cardiovascular AESIs.
While these safety results are concerning, we must not forget the other side of the equation. Unfortunately, the study doesn’t calculate composite estimates that also included the reduction in serious COVID infections, but we have such estimates for mortality.
One important limitation of both Fraiman’s and Benn’s studies is that they don’t distinguish the adverse reactions by age, comorbidities, or medical history. That isn’t their fault. Pfizer and Moderna haven’t released that information, so outside researchers don’t have access.
While everyone can get infected, children have a minuscule risk of COVID mortality. There is very limited safety data from the trials on children. If the risk of adverse reactions is the same as for adults, the harms outweigh the risks. Children shouldn’t receive these vaccines.
Older people above 70 have a much higher risk of COVID mortality than the population in the Fraiman study. If their risk of adverse reaction is the same, then the benefits outweigh the harms. Hence, older people who have never had COVID and aren’t yet vaccinated may benefit from these vaccines. However, we don’t know if they are better than the Johnson & Johnson and Astra-Zeneca vaccines.
It’s unclear from the clinical trial data whether the benefits outweigh the risks for working-age adults who haven’t been vaccinated and who haven’t already had COVID. This is true both historically for the original COVID variants and currently for the newer ones.
The Fraiman study analyzes data after the first and second doses. Both risks and benefits may differ for booster shots, but no randomized trial has properly evaluated the trade-off.
These results concern only the Pfizer and Moderna mRNA vaccines. Fraiman et al. didn’t analyze data on the adenovirus-vector vaccines marketed by Johnson & Johnson and Astra-Zeneca. Benn et al. found that they reduced all-cause mortality (RR=0.37, 95 percent CI:0.19-0.70), but nobody has used trial data to analyze AESIs for these vaccines.
Fraiman and colleagues have produced the best evidence yet regarding the overall safety of the mRNA vaccines. The results are concerning. It’s the responsibility of the manufacturers and FDA to ensure that benefits outweigh harms. They have failed to do so.
