27
$\begingroup$

In the last few weeks, Pfizer/BionTech, Moderna and AstraZeneca have each released preliminary estimates of the efficacy of their SARS-COV-2 vaccines.

But what do their respective efficacy percentages actually mean? Is the Pfizer/BioNTech vaccine 95% effective in any person? Or is the success of provoking the desired immune response limited to 95% of the population?

Put very simply: is it 100% effective in 95% of the population, or 95% effective in 100% of the population?

Improve this question
$\endgroup$
1
  • 6
    $\begingroup$ Summarizing, it means 95% fewer infections than the placebo group. How is not yet explained. Whether 95% become immune, or everyone becomes 95% tougher to infect. More importantly, the main purpose of corona vaccines is the old herd immunity. Like extinguishing a fire by hosing down all of the surrounding wood. See below for excellent detail and further reading. $\endgroup$ – thegreatwhatsit Nov 23 '20 at 18:40
27
$\begingroup$

Vaccine efficacy

Pfizer's target measures for efficacy (see the study on clinicaltrials.gov) seem to be:

Confirmed COVID-19 in Phase 2/3 participants without evidence of infection before vaccination

Confirmed COVID-19 in Phase 2/3 participants with and without evidence of infection before vaccination

From Pfizer's study plan (VE = vaccine efficacy):

VE will be estimated by 100 × (1 – IRR), where IRR is the calculated ratio of confirmed COVID-19 illness per 1000 person-years follow-up in the active vaccine group to the corresponding illness rate in the placebo group from 7 days after the second dose. VE will be analyzed using a beta-binomial model.

(note: they also have other time windows and checkpoints in their analysis plan; which one is reported will shift from press release to press release as they get more data. If you are interested in the details the study plan also describes the planned interim analyses, and some of the press releases discuss deviations they've made from their original interim plans in consultation with regulatory agencies)

This measure is called relative risk, and can be written like this:

%InfectedPerTimeplacebo = NInfectedplacebo / (NStudiedplacebo * AverageFollowUpTime)

%InfectedPerTimevaccine = NInfectedvaccine / (NStudiedvaccine * AverageFollowUpTime)

Efficacy aka VE = %Infectedvaccine / %Infectedplacebo

If you assume risks are the same in the placebo and vaccine groups (they "should" be, but might vary if, for example, people who experience vaccine side effects change their behavior) and the average follow-up time is the same (they should be approximately the same, because they are giving the vaccine and placebo to patients enrolled at the same time), this ratio would tell you that a vaccine efficacy of 95% means that if you took 20 people who would have had a positive test after the placebo, you would only expect 1 of them to test positive if they instead got the vaccine.

Population vs. individual statistics

Put very simply: is it 100% effective in 95% of the population, or 95% effective in 100% of the population?

These are population-based measures. They can't say anything about efficacy in particular individuals by these outcome measures. For example, there is no way to know from a study like this whether the vaccine is 100% effective in 95% of the population, or if it raises the infective dose in everyone by some amount which causes the number of people exposed to this critical viral dose in their environment to decrease by 95% (or some other effect with the same end result).

Other approaches like challenge studies, where vaccinated individuals (or animal models) are intentionally exposed to a certain dose of the virus, can help understand the individual effects of vaccination, as can indirect measures of immune response like antibody titers. These approaches have other drawbacks, however (safety, translating animal results to humans, translating a given immune response to an infection chance, etc).

Beyond just 'vaccine efficacy': disease severity, real-world efficacy

These particular outcomes also say nothing about disease severity. It could be that the people who do test positive despite getting the vaccine get just as sick as the sickest people who don't (interpretation would be that the vaccine protects mostly against mild illness). It could also be the reverse, and that people who get the vaccine and still test positive have a milder illness than they would have otherwise. Efficacy defined by this relative risk ratio does not say anything about this, it only compares positive vs negative rates.

An additional note: these numbers report efficacy in the trial environment. "Real-world" effectiveness (the same measure of effect, but in real world use rather than under trial conditions) might depend on other factors such as differences in the people enrolling in trials vs the general population (both in terms of things like age and preexisting conditions as well as behavior and exposure risks), failure to administer the vaccine properly (including improper storage), failure to complete both doses in timely fashion, etc. This measure of effectiveness also refers only to "primary" effectiveness. One would expect that if enough people were vaccinated, the effect on the population could far exceed the primary effectiveness, because not only do vaccinated individuals have a lower chance of infection, but everyone else in the population also has a lower risk if there are fewer people available to transmit the infection.

Improve this answer
$\endgroup$
4
  • 8
    $\begingroup$ Should also note that (as with flu vaccines, which are only around 40-60% effective for the individual) a major purpose of the vaccine is not to provide 100% protection to the person that receives it. It's to reduce the transmission rate so that the virus doesn't spread. $\endgroup$ – jamesqf Nov 23 '20 at 18:08
  • 6
    $\begingroup$ @jamesqf Meant to include that in my final paragraph but it slipped my mind; included now. $\endgroup$ – Bryan Krause♦ Nov 23 '20 at 18:13
  • $\begingroup$ "Efficacy aka VE = %Infected_vaccine / %Infected_placebo" -- did you mean IRR here? $\endgroup$ – aland Nov 25 '20 at 11:00
  • 1
    $\begingroup$ AFAIK the Pfizer vaccine showed that the disease severity was also lowered in the group that did catch Corona despite being vaccinated; none of those developed a severe form. The control group apparently had a few people hospitalized. $\endgroup$ – MSalters Nov 26 '20 at 14:22
10
$\begingroup$

It means protection against the virus brought to you by the vaccination.

Around 45.000 people participate in the trial; 50% of these are vaccinated with the trial vaccine and 50% receive a placebo. Additionally, you exclude people from the trial who had had COVID-19 before. Because infecting people with a potentially deadly disease is not possible ethically, you watch both groups for occurrence of the disease.

I have numbers only for the Biontech/Pfizer vaccine, but the principle is the same for the others as well. In this trial 170 cases of COVID-19 were seen: 162 in the placebo group and only 8 in the vaccine group. If the vaccine were to do nothing at all, then you would expect to see similar numbers in both groups, which means that these 8 cases from the vaccine group represent a reduction (or efficacy) of about 95%.

With the vaccine, only 5% will catch the disease, while there is no protection for unvaccinated persons. For the numbers, see the press release from Pfizer and the database entry on clinical trials for this trial.

Improve this answer
$\endgroup$
6
  • 4
    $\begingroup$ To further clarify the statement that "with the vaccine, only 5% will catch the disease", this means that 5% of vaccinated people who would have otherwise caught the disease will become infected, not that 5% of all vaccinated individuals will catch the disease (only about 0.01% of vaccinated people did). $\endgroup$ – Nuclear Hoagie Nov 23 '20 at 17:46
  • 4
    $\begingroup$ @NuclearHoagie Exactly. Without the vaccine you would expect similar numbers of infections in both groups, which is reduced by 95%. $\endgroup$ – Chris♦ Nov 23 '20 at 20:00
  • $\begingroup$ How do you get .05 out of this though? 166/168 is about .988. $\endgroup$ – Tim kinsella Mar 3 at 7:19
  • 1
    $\begingroup$ @Timkinsella You are right, I somehow added two wrong numbers, which are corrected now. It should be 8 people for the vaccinated group vs. 162 in the control group. See the original publication. Thanks for pointing out. $\endgroup$ – Chris♦ Mar 3 at 7:37
  • $\begingroup$ @Chris There is one more “2” to change to “8” in the third paragraph. I tried to change it but it’s below the character limit for edits at my rep level, apparently. $\endgroup$ – Tim kinsella Mar 3 at 18:42
5
$\begingroup$

Saying that a vaccine is $95\%$ efficacious is neither claiming that it works $95\%$ of the time nor that it protects $95\%$ of its recipients.†

Rather, a $95\%$ efficacy means that—during the clinical trial in which half the subjects had been vaccinated and half hadn't—among those who eventually developed the disease, the ratio of those who had been vaccinated to those who hadn't is $5 : 100.$

efficacy vaccinated : unvaccinated

among those who contracted the disease
$100\%$ $0 : 100$
$95\%$ $5 : 100$
$70\%$ $30 : 100$
$50\%$ $50 : 100$
$0\%$ $100 : 100$

(the vaccine makes no difference)
$-20\%$ $120 : 100$

(the vaccine makes you more susceptible to the disease)

Think of vaccine efficacy as a measure of the increase in protection conferred by the vaccine during the clinical trial҂—not as an indication of how frequently the vaccine works (either across time, or among the vaccinated population).

If the clinical trial was short (as was the case with the Pfizer-BioNTech vaccine), then the published vaccine efficacy could not have taken immunity duration into account. Therefore the figure may vary—perhaps drastically—as the subjects continue to be monitored over the coming months/years.

† Such claims assume how the efficacy is distributed (e.g., that it works discretely—either works or doesn't—across recipients). And certainly, 95% efficacy is not the recipient's probability of not contracting the disease, as that depends on their health and exposure, the disease evolution, and other factors.

҂ Technically, vaccine efficacy is the proportionate reduction in disease occurrence conferred by the vaccine during the clinical trial.

P.S. Mathematically, $$\text{vaccine efficacy}=\frac{U-V}U=1-R,$$

  • where $U$ and $V$ are the respective numbers of unvaccinated and vaccinated subjects who contracted the disease (exactly half the subjects had gotten the vaccination), and
  • $R=\frac VU$ is the relative risk (the right-side column in the above table).

P.P.S. For the Pfizer-BioNTech trial (in which $43661$ subjects were split evenly between the placebo and vaccine groups), $U=162$ and $V=8,$ so \begin{align}\text{vaccine efficacy}&=\frac{162-8}{162}\\&=95.1\%.\end{align}

P.P.P.S. Note that the Pfizer-BioNTech trial checked only for symptomatic disease, i.e., the $U=162$ and $V=8$ do not include asymptomatic cases of COVID-19 among the study subjects.

Improve this answer
$\endgroup$
0
3
$\begingroup$

Vaccine efficacy is the percent change in the percent of individuals who test positive in the vaccinated group versus the placebo group in a trial. The CDC website explains it here

So a 95% efficacy means the vaccinated group had 95% less percentage of people test positive than in the placebo group. So if the placebo group had a 20% positive rate, the vaccinated group had a 1% positive rate (5% of 20%). If this 95% efficacy is deemed to be statistically significant, it likely means that the positive rate in the vaccinated portion of the population will be 5% of the positive rate in the unvaccinated population.

Normally, if 100% of the population takes a vaccine, a 60% vaccine efficacy is considered enough to cause enough decrease in positivity rate to stop the spread of COVID-19 so 95% is really good. This paper goes into what the vaccine efficacy needs to be if less than 100% of the population takes the vaccine.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I'm sure you'll have seen the recent results of the Oxford vaccine by now! The headlines stated 70% efficacy at first, with the intended regimen of two full doses two week(?) apart, but found (by accident, apparently) that if the first dose was a half dose, the efficacy rose to 90%. $\endgroup$ – drkvogel Nov 24 '20 at 3:26
  • $\begingroup$ More intriguingly, Professor Jonathan Van-Tam, Deputy Chief Medical Officer for England, said today that of the 24,000 (I think) trial participants who were given the vaccine in either regimen, none of them needed to be admitted to hospital. Does this, I wonder, mean that although the efficacy in terms of not contracting the virus at all was 70% or 90%, the people that did get the virus had milder symptoms than usual, and thus the efficacy in terms of preventing hospitalisation and death is closer to 100%? What would the name for that be? $\endgroup$ – drkvogel Nov 24 '20 at 3:30

Not the answer you're looking for? Browse other questions tagged or ask your own question.