What does vaccine efficacy mean?

Question

In the last few weeks, Pfizer/BionTech, Moderna and AstraZeneca have each released preliminary estimates of the efficacy of their SARS-COV-2 vaccines.

But what do their respective efficacy percentages actually mean? Is the Pfizer/BioNTech vaccine 95% effective in any person? Or is the success of provoking the desired immune response limited to 95% of the population?

Put very simply: is it 100% effective in 95% of the population, or 95% effective in 100% of the population?

Answer 1

Vaccine efficacy

Pfizer's target measures for efficacy (see the study on clinicaltrials.gov) seem to be:

Confirmed COVID-19 in Phase 2/3 participants without evidence of infection before vaccination

Confirmed COVID-19 in Phase 2/3 participants with and without evidence of infection before vaccination

From Pfizer's study plan (VE = vaccine efficacy):

VE will be estimated by 100 × (1 – IRR), where IRR is the calculated ratio of confirmed COVID-19 illness per 1000 person-years follow-up in the active vaccine group to the corresponding illness rate in the placebo group from 7 days after the second dose. VE will be analyzed using a beta-binomial model.

(note: they also have other time windows and checkpoints in their analysis plan; which one is reported will shift from press release to press release as they get more data. If you are interested in the details the study plan also describes the planned interim analyses, and some of the press releases discuss deviations they've made from their original interim plans in consultation with regulatory agencies)

This measure is called relative risk, and can be written like this:

%InfectedPerTime_placebo = NInfected_placebo / (NStudied_placebo * AverageFollowUpTime)

%InfectedPerTime_vaccine = NInfected_vaccine / (NStudied_vaccine * AverageFollowUpTime)

Efficacy aka VE = %Infected_vaccine / %Infected_placebo

If you assume risks are the same in the placebo and vaccine groups (they "should" be, but might vary if, for example, people who experience vaccine side effects change their behavior) and the average follow-up time is the same (they should be approximately the same, because they are giving the vaccine and placebo to patients enrolled at the same time), this ratio would tell you that a vaccine efficacy of 95% means that if you took 20 people who would have had a positive test after the placebo, you would only expect 1 of them to test positive if they instead got the vaccine.

Population vs. individual statistics

Put very simply: is it 100% effective in 95% of the population, or 95% effective in 100% of the population?

These are population-based measures. They can't say anything about efficacy in particular individuals by these outcome measures. For example, there is no way to know from a study like this whether the vaccine is 100% effective in 95% of the population, or if it raises the infective dose in everyone by some amount which causes the number of people exposed to this critical viral dose in their environment to decrease by 95% (or some other effect with the same end result).

Other approaches like challenge studies, where vaccinated individuals (or animal models) are intentionally exposed to a certain dose of the virus, can help understand the individual effects of vaccination, as can indirect measures of immune response like antibody titers. These approaches have other drawbacks, however (safety, translating animal results to humans, translating a given immune response to an infection chance, etc).

Beyond just 'vaccine efficacy': disease severity, real-world efficacy

These particular outcomes also say nothing about disease severity. It could be that the people who do test positive despite getting the vaccine get just as sick as the sickest people who don't (interpretation would be that the vaccine protects mostly against mild illness). It could also be the reverse, and that people who get the vaccine and still test positive have a milder illness than they would have otherwise. Efficacy defined by this relative risk ratio does not say anything about this, it only compares positive vs negative rates.

An additional note: these numbers report efficacy in the trial environment. "Real-world" effectiveness (the same measure of effect, but in real world use rather than under trial conditions) might depend on other factors such as differences in the people enrolling in trials vs the general population (both in terms of things like age and preexisting conditions as well as behavior and exposure risks), failure to administer the vaccine properly (including improper storage), failure to complete both doses in timely fashion, etc. This measure of effectiveness also refers only to "primary" effectiveness. One would expect that if enough people were vaccinated, the effect on the population could far exceed the primary effectiveness, because not only do vaccinated individuals have a lower chance of infection, but everyone else in the population also has a lower risk if there are fewer people available to transmit the infection.

Answer 2

It means protection against the virus brought to you by the vaccination.

Around 45.000 people participate in the trial; 50% of these are vaccinated with the trial vaccine and 50% receive a placebo. Additionally, you exclude people from the trial who had had COVID-19 before. Because infecting people with a potentially deadly disease is not possible ethically, you watch both groups for occurrence of the disease.

I have numbers only for the Biontech/Pfizer vaccine, but the principle is the same for the others as well. In this trial 170 cases of COVID-19 were seen: 162 in the placebo group and only 8 in the vaccine group. If the vaccine were to do nothing at all, then you would expect to see similar numbers in both groups, which means that these 8 cases from the vaccine group represent a reduction (or efficacy) of about 95%.

With the vaccine, only 5% will catch the disease, while there is no protection for unvaccinated persons. For the numbers, see the press release from Pfizer and the database entry on clinical trials for this trial.

Answer 3

Saying that a vaccine is $95\%$ efficacious is neither claiming that it works $95\%$ of the time nor that it protects $95\%$ of its recipients.^†

Rather, a $95\%$ efficacy means that—during the clinical trial in which half the subjects had been vaccinated and half hadn't—among those who eventually developed the disease, the ratio of those who had been vaccinated to those who hadn't is $5 : 100.$

efficacy	among those who contracted the disease, the ratio of vaccinated to unvaccinated participants	(if effectiveness=efficacy) in any given situation, one is ___ times as likely to be infected without the vaccine than with
$100\%$	$0 : 100$	$\infty$
$95\%$	$5 : 100$	$20$
$60\%$	$40 : 100$	$2.5$
$50\%$	$50 : 100$	$2$
$0\%$	$100 : 100$	$1$ (the vaccine has no effect)
$-20\%$	$120 : 100$	$0.833$ (the vaccine makes you more susceptible to the disease)

Think of vaccine efficacy as a measure of the increase in protection conferred by the vaccine during the clinical trial^҂—not as an indication of how frequently the vaccine successfully blocks infection (whether across time, or among the vaccinated population).

If the clinical trial was short (as was the case with the Pfizer-BioNTech vaccine), then the published vaccine efficacy could not have taken immunity duration^җ. into account. Therefore the figure may vary—perhaps significantly—as the subjects continue to be monitored over the coming months/years.

^{† Such claims assume how the efficacy is distributed (e.g., that it works discretely—either works or doesn't—across recipients). And certainly, 95% efficacy is not the recipient's probability of not contracting the disease, as that depends on their health and exposure, the disease evolution, and other factors.}

^{҂ Technically, vaccine efficacy is the proportionate reduction in disease occurrence conferred by the vaccine during the clinical trial.}

^{җFollow-up data since this Answer was written indicates what the Pfizer-BioNTech vaccine's real-world effectiveness wanes by $10-35$ percentage points six months following vaccination. These figures are complicated by the predominance of a more infectious variant (Delta) which wasn't around during the clinical trial.}

P.S. Mathematically, $$\text{vaccine efficacy}=\frac{U-V}U=1-R,$$

• where $U$ and $V$ are the respective numbers of unvaccinated and vaccinated subjects who contracted the disease (exactly half the subjects had gotten the vaccination), and
• $R=\frac VU$ is the relative risk (the right-side column in the above table).

P.P.S. For the Pfizer-BioNTech trial (in which $43661$ subjects were split evenly between the placebo and vaccine groups), $U=162$ and $V=8,$ so \begin{align}\text{vaccine efficacy}&=\frac{162-8}{162}\\&=95.1\%.\end{align}

P.P.P.S. Note that the Pfizer-BioNTech trial checked only for symptomatic disease, i.e., the $U=162$ and $V=8$ do not include asymptomatic cases of COVID-19 among the study subjects.

Answer 4

Vaccine efficacy is the percent change in the percent of individuals who test positive in the vaccinated group versus the placebo group in a trial. The CDC website explains it here

So a 95% efficacy means the vaccinated group had 95% less percentage of people test positive than in the placebo group. So if the placebo group had a 20% positive rate, the vaccinated group had a 1% positive rate (5% of 20%). If this 95% efficacy is deemed to be statistically significant, it likely means that the positive rate in the vaccinated portion of the population will be 5% of the positive rate in the unvaccinated population.

Normally, if 100% of the population takes a vaccine, a 60% vaccine efficacy is considered enough to cause enough decrease in positivity rate to stop the spread of COVID-19 so 95% is really good. This paper goes into what the vaccine efficacy needs to be if less than 100% of the population takes the vaccine.