Post by DrSchadenfreude on Aug 25, 2020 9:34:48 GMT -5
F.D.A. ‘Grossly Misrepresented’ Blood Plasma Data, Scientists Say
Many experts — including a scientist who worked on the Mayo Clinic study — were bewildered about where a key statistic came from.
At a news conference on Sunday announcing the emergency approval of blood plasma for hospitalized Covid-19 patients, President Trump and two of his top health officials cited the same statistic: that the treatment had reduced deaths by 35 percent.
Mr. Trump called it a “tremendous” number. His health and human services secretary, Alex M. Azar II, a former pharmaceutical executive, said, “I don’t want you to gloss over this number.” And Dr. Stephen M. Hahn, the commissioner of the Food and Drug Administration, said 35 out of 100 Covid-19 patients “would have been saved because of the administration of plasma.”
But scientists were taken aback by the way the administration framed this data, which appeared to have been calculated based on a small subgroup of hospitalized Covid-19 patients in a Mayo Clinic study: those who were under 80 years old, not on ventilators and received plasma known to contain high levels of virus-fighting antibodies within three days of diagnosis.
What’s more, many experts — including a scientist who worked on the Mayo Clinic study — were bewildered about where the statistic came from. The number was not mentioned in the official authorization letter issued by the agency, nor was it in a 17-page memo written by F.D.A. scientists. It was not in an analysis conducted by the Mayo Clinic that has been frequently cited by the administration.
“For the first time ever, I feel like official people in communications and people at the F.D.A. grossly misrepresented data about a therapy,” said Dr. Walid Gellad, who leads the Center for Pharmaceutical Policy and Prescribing at the University of Pittsburgh.
It is especially worrisome, he said, given concerns over how Mr. Trump has appeared to politicize the process of approving treatments and vaccines for the coronavirus. Over the next couple of months, as data emerges from vaccine clinical trials, the safety of potentially millions of people will rely on the scientific judgment of the F.D.A. “That’s a problem if they’re starting to exaggerate data,” Dr. Gellad said. “That’s the big problem.”
When asked where the 35 percent figure came from, an agency spokeswoman initially directed a reporter to a graph of survival statistics buried in the Trump administration’s application for emergency authorization. The chart, analyzing the same tiny subset of Mayo Clinic study patients, did not include numerical figures, but it appeared to indicate a 30-day survival probability of about 63 percent in patients who received plasma with a low level of antibodies, compared with about 76 percent in those who received a high level of antibodies.
On Monday, Dr. Peter Marks, the director of F.D.A.’s center for biologics, evaluation and research, said that the agency reviewed published studies of plasma and conducted its own analysis of data from the Mayo Clinic’s program of hospitalized patients who received plasma. Although the size of the benefit varied, he said in a statement, “there appears to be roughly a 35 percent relative improvement in the survival rates of patients” who received the plasma with higher versus lower levels of antibodies.
He added: “Given the safety profile observed, the totality of evidence regarding potential efficacy more than adequately met the ‘may be effective’ standard for granting an Emergency Use Authorization.”
Convalescent plasma, the pale yellow liquid left over after blood is stripped of its red and white cells, has been the subject of months of enthusiasm from scientists, celebrities and Mr. Trump, part of the administration’s push for coronavirus treatments as a stopgap while pharmaceutical companies race to complete dozens of clinical trials for coronavirus vaccines.
Although there have been some positive signs that it can reduce deaths in Covid-19 patients, no randomized trials have shown that it works. A popular access program set up this spring by the F.D.A. and run by the Mayo Clinic has provided the treatment to more than 70,000 people, but it has also, some researchers said, hindered enrollment in the more rigorous randomized trials that could definitively prove whether it works. The emergency authorization released on Sunday broadens that access.
Statisticians and scientists said that Dr. Hahn, in saying at the news conference that 35 out of 100 sick Covid-19 patients would have been saved by receiving plasma, appeared to have overstated the benefits.
Dr. Robert Califf, who was F.D.A. commissioner under President Barack Obama, said on Twitter on Sunday that Dr. Hahn should correct his statement.
It would be good for Steve to publish a correction. I'm sure he meant to correctly state the translation from relative effect to absolute effect. @erictopol @scottgottliebmd @us_FDA t.co/AWEJaGDOkn
— Robert M Califf (@califf001) August 24, 2020
The publicly released data from the Mayo Clinic shows that, among the larger group of more than 35,000 patients, when plasma was given within three day of diagnosis, the death rate was about 22 percent, compared with 27 percent when it was given four or more days after diagnosis.
Dr. Eric Topol, a professor of molecular medicine at Scripps Research in La Jolla, Calif. and a longtime expert in clinical trials, said that convalescent plasma has not yet shown the benefit that Dr. Hahn described — and that he should issue a correction.
“He needs to come out with that, and until he does, he has no credibility as an F.D.A. commissioner,” Dr. Topol said.
On Monday night, after this article was published, Dr. Hahn tweeted about the scientific response to his remarks, saying the “criticism is entirely justified.” He clarified that his earlier statements suggested an absolute reduction in risk, instead of the relative risk of a certain group of patients compared with another.
I have been criticized for remarks I made Sunday night about the benefits of convalescent plasma. The criticism is entirely justified. What I should have said better is that the data show a relative risk reduction not an absolute risk reduction.
— Dr. Stephen M. Hahn (@stevefda) August 25, 2020
On Sunday night, an agency spokeswoman had posted a chart on Twitter claiming that plasma “has shown to be beneficial” for 35 percent of patients — neglecting to mention that the figure was based on a subset of a subset of the data.
Convalescent plasma has shown to be beneficial for 35% of patients. This risk reduction figure - shown in chart below - is from @mayoclinic data from expanded access program that was analyzed by FDAA for the emergency use authorization announced today. pic.twitter.com/UNAWrhHa3p
— Emily J. Miller - FDA (@fdaspox) August 24, 2020
In an interview on Monday, one of the Mayo Clinic study’s main authors, Dr. Arturo Casadevall of Johns Hopkins University in Baltimore, said he was also stumped. “Do I know where the 35 percent comes from?” he said. “No.”
Dr. Casadevall said that the F.D.A. had conducted its own analyses of the data, and that the paper he and his colleagues posted this month to a so-called preprint server — before it has been peer reviewed by a medical journal — contained only a portion of the total available data.
Dr. Casadevall said focusing on the 35 percent number distracted from the broader takeaway: that convalescent plasma shows promise, at a time when doctors have few other options. “I think that I would not focus so much on that — people can say things in many different ways, but I think that one has to look at the data,” he said. “I think the important thing is that all the indicators show a reduction in mortality.”
Emily R. Smith, an epidemiologist at The George Washington University Milken Institute School of Public Health in Washington, D.C., agreed that the issue was a distraction, but for a different reason.
“We’re trying to track down numbers and preprints and Twitter slides, and that’s a big distraction from the bigger issue of — do we think there’s good evidence to suggest this is safe and effective?” she said.
She and others, including the F.D.A. in the emergency authorization itself, have said that existing data — from the Mayo Clinic and other non-randomized studies — cannot replace rigorous trials that will more definitively indicate whether and in what groups of patients the treatment is effective. On Monday, the chief scientist for the World Health Organization, Dr. Soumya Swaminathan, said at a news conference in Geneva that only a few of these convalescent plasma trials had reported findings, and that the trials had been relatively small. “The results in some cases point to some benefit but have not been conclusive,” she said. “At the moment it’s still very low-quality evidence.”
Many experts — including a scientist who worked on the Mayo Clinic study — were bewildered about where a key statistic came from.
August 24, 2020
Mr. Trump called it a “tremendous” number. His health and human services secretary, Alex M. Azar II, a former pharmaceutical executive, said, “I don’t want you to gloss over this number.” And Dr. Stephen M. Hahn, the commissioner of the Food and Drug Administration, said 35 out of 100 Covid-19 patients “would have been saved because of the administration of plasma.”
But scientists were taken aback by the way the administration framed this data, which appeared to have been calculated based on a small subgroup of hospitalized Covid-19 patients in a Mayo Clinic study: those who were under 80 years old, not on ventilators and received plasma known to contain high levels of virus-fighting antibodies within three days of diagnosis.
What’s more, many experts — including a scientist who worked on the Mayo Clinic study — were bewildered about where the statistic came from. The number was not mentioned in the official authorization letter issued by the agency, nor was it in a 17-page memo written by F.D.A. scientists. It was not in an analysis conducted by the Mayo Clinic that has been frequently cited by the administration.
“For the first time ever, I feel like official people in communications and people at the F.D.A. grossly misrepresented data about a therapy,” said Dr. Walid Gellad, who leads the Center for Pharmaceutical Policy and Prescribing at the University of Pittsburgh.
It is especially worrisome, he said, given concerns over how Mr. Trump has appeared to politicize the process of approving treatments and vaccines for the coronavirus. Over the next couple of months, as data emerges from vaccine clinical trials, the safety of potentially millions of people will rely on the scientific judgment of the F.D.A. “That’s a problem if they’re starting to exaggerate data,” Dr. Gellad said. “That’s the big problem.”
When asked where the 35 percent figure came from, an agency spokeswoman initially directed a reporter to a graph of survival statistics buried in the Trump administration’s application for emergency authorization. The chart, analyzing the same tiny subset of Mayo Clinic study patients, did not include numerical figures, but it appeared to indicate a 30-day survival probability of about 63 percent in patients who received plasma with a low level of antibodies, compared with about 76 percent in those who received a high level of antibodies.
On Monday, Dr. Peter Marks, the director of F.D.A.’s center for biologics, evaluation and research, said that the agency reviewed published studies of plasma and conducted its own analysis of data from the Mayo Clinic’s program of hospitalized patients who received plasma. Although the size of the benefit varied, he said in a statement, “there appears to be roughly a 35 percent relative improvement in the survival rates of patients” who received the plasma with higher versus lower levels of antibodies.
He added: “Given the safety profile observed, the totality of evidence regarding potential efficacy more than adequately met the ‘may be effective’ standard for granting an Emergency Use Authorization.”
Convalescent plasma, the pale yellow liquid left over after blood is stripped of its red and white cells, has been the subject of months of enthusiasm from scientists, celebrities and Mr. Trump, part of the administration’s push for coronavirus treatments as a stopgap while pharmaceutical companies race to complete dozens of clinical trials for coronavirus vaccines.
Although there have been some positive signs that it can reduce deaths in Covid-19 patients, no randomized trials have shown that it works. A popular access program set up this spring by the F.D.A. and run by the Mayo Clinic has provided the treatment to more than 70,000 people, but it has also, some researchers said, hindered enrollment in the more rigorous randomized trials that could definitively prove whether it works. The emergency authorization released on Sunday broadens that access.
Statisticians and scientists said that Dr. Hahn, in saying at the news conference that 35 out of 100 sick Covid-19 patients would have been saved by receiving plasma, appeared to have overstated the benefits.
Dr. Robert Califf, who was F.D.A. commissioner under President Barack Obama, said on Twitter on Sunday that Dr. Hahn should correct his statement.
It would be good for Steve to publish a correction. I'm sure he meant to correctly state the translation from relative effect to absolute effect. @erictopol @scottgottliebmd @us_FDA t.co/AWEJaGDOkn
— Robert M Califf (@califf001) August 24, 2020
The publicly released data from the Mayo Clinic shows that, among the larger group of more than 35,000 patients, when plasma was given within three day of diagnosis, the death rate was about 22 percent, compared with 27 percent when it was given four or more days after diagnosis.
Dr. Eric Topol, a professor of molecular medicine at Scripps Research in La Jolla, Calif. and a longtime expert in clinical trials, said that convalescent plasma has not yet shown the benefit that Dr. Hahn described — and that he should issue a correction.
“He needs to come out with that, and until he does, he has no credibility as an F.D.A. commissioner,” Dr. Topol said.
On Monday night, after this article was published, Dr. Hahn tweeted about the scientific response to his remarks, saying the “criticism is entirely justified.” He clarified that his earlier statements suggested an absolute reduction in risk, instead of the relative risk of a certain group of patients compared with another.
I have been criticized for remarks I made Sunday night about the benefits of convalescent plasma. The criticism is entirely justified. What I should have said better is that the data show a relative risk reduction not an absolute risk reduction.
— Dr. Stephen M. Hahn (@stevefda) August 25, 2020
On Sunday night, an agency spokeswoman had posted a chart on Twitter claiming that plasma “has shown to be beneficial” for 35 percent of patients — neglecting to mention that the figure was based on a subset of a subset of the data.
Convalescent plasma has shown to be beneficial for 35% of patients. This risk reduction figure - shown in chart below - is from @mayoclinic data from expanded access program that was analyzed by FDAA for the emergency use authorization announced today. pic.twitter.com/UNAWrhHa3p
— Emily J. Miller - FDA (@fdaspox) August 24, 2020
In an interview on Monday, one of the Mayo Clinic study’s main authors, Dr. Arturo Casadevall of Johns Hopkins University in Baltimore, said he was also stumped. “Do I know where the 35 percent comes from?” he said. “No.”
Dr. Casadevall said that the F.D.A. had conducted its own analyses of the data, and that the paper he and his colleagues posted this month to a so-called preprint server — before it has been peer reviewed by a medical journal — contained only a portion of the total available data.
Dr. Casadevall said focusing on the 35 percent number distracted from the broader takeaway: that convalescent plasma shows promise, at a time when doctors have few other options. “I think that I would not focus so much on that — people can say things in many different ways, but I think that one has to look at the data,” he said. “I think the important thing is that all the indicators show a reduction in mortality.”
Emily R. Smith, an epidemiologist at The George Washington University Milken Institute School of Public Health in Washington, D.C., agreed that the issue was a distraction, but for a different reason.
“We’re trying to track down numbers and preprints and Twitter slides, and that’s a big distraction from the bigger issue of — do we think there’s good evidence to suggest this is safe and effective?” she said.
She and others, including the F.D.A. in the emergency authorization itself, have said that existing data — from the Mayo Clinic and other non-randomized studies — cannot replace rigorous trials that will more definitively indicate whether and in what groups of patients the treatment is effective. On Monday, the chief scientist for the World Health Organization, Dr. Soumya Swaminathan, said at a news conference in Geneva that only a few of these convalescent plasma trials had reported findings, and that the trials had been relatively small. “The results in some cases point to some benefit but have not been conclusive,” she said. “At the moment it’s still very low-quality evidence.”
