Pfizer vaccine results are promising, but lack of data ‘very concerning,’ experts say

The world cheered this week after pharmaceutical companies Pfizer and BioNTech announced early findings from their phase three trial for a frontrunner COVID-19 vaccine. On November 9, the companies issued a news release claiming their vaccine candidate effectively prevented infection with the coronavirus—welcome news after soaring global cases have prompted new shutdowns and warnings of intensive care units filled to capacity.

But even more stunning was the extent the vaccine exceeded industry expectations. According to their press release, data reviewed by an independent panel of experts show the vaccine to be 90-percent effective, meaning nine out of 10 recipients experienced some sort of benefit—far surpassing the 50-percent benchmark set by health overseers such as the U.S. Food and Drug Administration and the World Health Organization.

The news was unprecedented because it appears to be a vindication of the mRNA technology that so far has only been proven in animal models. But it was also surprising because the companies took the unusual step of releasing some of the results—what they called an “interim report”—before the rest of the data was available.

(Related: The COVID-19 "infodemic," explained.)

“As a scientist, you’d want to see actual data from a clinical trial to really know what the actual interpretation of the results are,” says Aliasger K. Salem, chair of pharmaceutical sciences at the University of Iowa.

Several experts say they’re concerned that the public is getting an incomplete picture about the vaccine’s success that doesn’t reveal critical information, such as which demographic groups it protected and whether it was from a mild or severe form of the virus. There’s also the real possibility that the 90-percent figure could change as the trial ticks on and investigators collect more results. Plus, the unpublished results have not been peer-reviewed or even released as a preliminary preprint.

“The lack of data is very concerning,” adds Peter Doshi, associate professor of pharmaceutical health services research at the University of Maryland School of Pharmacy. “All we have right now is a headline by Pfizer.”

Yet others praised the decision—saying that it’s unlikely the 90-percent result would change significantly in the meantime. “It probably will shift, but it might be 87. But given the numbers they have in the trial, it can’t be 50 percent,” says Robert Wachter, chair of the University of California, San Francisco, department of medicine who studies patient safety. “It’s clearly in the range of 90.”

Nuts and Bolts

Here’s how the Pfizer-BioNTech vaccine trial works. The third and final phase began in July, when investigators evenly assigned 43,538 participants to two groups: one to receive the vaccine being studied, and another to get an inactive substance known as a placebo. According to the trial’s protocol, the vaccine comes in two doses, which are given three weeks apart. The researchers waited at least seven days after the second dose had been administered to see who contracted COVID-19 in real life, assuming each group practiced comparable risk-taking behaviors.

To keep the process unbiased, drug companies are typically “blinded” during phase three to what’s happening on a day-to-day basis in terms of results. But the COVID-19 vaccine trial comes with exceptions—milestones based on how many cases have been recorded among the volunteers. Once hit, the independent panel—known as a data monitoring committee—can tell the companies how things are going. Pfizer and BioNTech could make this announcement now because the trial reached one of these milestones, documenting 94 cases of COVID-19.

Yet the preliminary data doesn’t reveal which of these infected patients received the vaccine and whom it failed. It also doesn’t involve all the trial participants but rather an undefined subset, though Wachter says his best guess from the press release is the vaccine group recorded about nine COVID-19 cases, while the placebo arm experienced about 90.

The risk of not reporting these complete results is that the headline could send a misleading message that “the vaccine provides greater benefit than it really does,” says Doshi, who studies the drug approval process. “Pfizer hasn’t released anything indicating that this 90-percent effective statistic applies to those most frail, like older adults or those in nursing homes who are at highest risk.”

(Related: A "herd mentality" can’t stop the COVID-19 pandemic. Neither can a weak vaccine.)

You May Also Like

SCIENCE

Here’s the latest on COVID-19 vaccines

SCIENCE

What is 'immune amnesia?' This long-term side effect of measles is newly relevant.

SCIENCE

Why does COVID-19 cause brain fog? Scientists may finally have an answer.

It’s also unclear if the vaccine simply prevented symptoms in mild or moderate cases, or if it reduced severe outcomes. Such information is critical because a vaccine ideally would reduce the number of deaths, hospitalization, and ICU visits. “The studies should give us a sense of severity, but they’re only looking at symptoms,” says Maria Elena Bottazzi, co-director of the Texas Children’s Hospital Center for Vaccine Development in Houston.

It’s also unknown whether any of the 94 coronavirus patients had underlying risk factors. That data matters because state and county officials are prioritizing high-risk and socially vulnerable groups during the first distribution waves. “It’s not clear in the trial who got the vaccine,” says Paul Offit, director of the Vaccine Education Center at Children’s Hospital of Philadelphia and a member of the FDA’S COVID-19 advisory committee. “You wonder whether those groups of people were adequately represented.”

The home stretch?

News of the announcement boosted stock markets, helping reverse a downturn witnessed over recent weeks. But this reaction also raised questions of why Pfizer and BioNTech decided to release this interim data? When asked if Pfizer was under pressure from stockholders, Jerica Pitts, Pfizer’s director of global media relations, said the company was not required to release the report but it had always planned to do so at its discretion.

“We had always said that we would share interim results given how needed this breakthrough could be,” Pitts stated in an email to National Geographic. “We have shared that these [results] are not conclusive and only observe those participants who have not previously been infected or contracted COVID-19.”

In their press release, the companies said the next step was to collect more safety data and apply to the FDA for an emergency use authorization in the third week of November. The agency requires that a vaccine-maker provide two months’ worth of safety data for at least half its phase three participants before such an authorization will be considered.

Yet recent meetings of vaccine advisory committees at the FDA and CDC indicated that regulators were considering the idea of swapping an emergency authorization with what’s known as an “expanded access,” or “compassionate use” license.

Such a step would limit how many people could receive a frontrunner vaccine, which would come with pros and cons. It would still allow access for dire situations, such as for frontline health-care workers and vulnerable communities, but there’d be reduced access for the general public. By contrast, an emergency authorization would mean no one could be denied the vaccine, which would essentially eliminate the ability to complete clinical trials for a vaccine candidate. People in placebo groups could request the vaccine, and scientists could no longer judge in a systematic way if the drug was truly working or safe.

Offit says a change in licensing goals might also be a positive step in encouraging more public confidence, given that some previous emergency use decisions, such as the one for the antimalarial drug hydroxychloroquine, have been controversial. “It’s better language,” says Offit, explaining that some emergency authorizations have been seen as being politically motivated on inadequately tested treatments. “I don’t think people should be suspicious of this vaccine.”

Some experts questioned why the companies chose to issue a press release before the safety data were ready. “There was no need,” says Doshi. “We could have waited two weeks for more complete data.”

But Wachter counters that any concerns about patient safety probably would have surfaced by now. “It would be nice to have more information on symptoms and how many were hospitalized. But the data is good enough so that I’m not worried about people over-interpreting the results,” he says. He adds that the data monitoring committees will stop a trial for suspicious or significant side effects. So far, that’s happened for three other COVID-19 vaccine trials, but not with Pfizer’s.

In fact, Wachter suggests the companies had a moral obligation to release early good news during this particularly grim phase of the pandemic. The announcement's influence on people's perceptions of the vaccine and the stock market isn't problematic as long as the interim report is truthful and suitably caveated, he says.

“This is incredibly promising news, and it seems quite unlikely that more detailed results will lead to a different conclusion than we're seeing now,” he says. “You could wait until they cross every last t. Then there’s another argument that it’s unethical to sit on the data any longer than you need to.”