Precision medicine may be health care’s next big thing, but we don’t yet know how to evaluate precision drugs

US health regulators have a lot of work to do.
US health regulators have a lot of work to do.
Image: Reuters/Carlos Barria
We may earn a commission from links on this page.

Finally, there’s a drug available to treat Duchenne muscular dystrophy (DMD), a rare, fatal genetic muscle-wasting disease that grips roughly 10,000 people, mostly boys, in the US, who are unlikely to live into their thirties and typically lose the ability to walk in adolescence.

On September 19, the Food and Drug Administration (FDA) approved Exondys 51 (eteplirsen) after reviewing a clinical trial and hearing words of support from 50 advocates on behalf of the drug, including families with children living with the disease, experts on the illness at UCLA, and even a few politicians, like Florida senator and former presidential candidate Marco Rubio.

Duchenne families and advocates were ecstatic at the news. Sarepta, the Cambridge, Massachusetts-based biotech firm that developed the drug, saw its stock more than double. There was just one minor issue: The science suggests the drug doesn’t work.

An advisory committee impaneled by the FDA in April to weigh the clinical data wasn’t convinced and voted against Exondys 51’s approval. But that panel was overruled by Janet Woodcock, director of the FDA’s Center for Drug Evaluation and Research. Woodcock greenlit Exondys 51 immediately—even though it will only benefit the 13% of Duchenne patients who have the specific genetic mutation the therapy targets—in part because there are no other options for DMD patients.

Her decision kicked off a skirmish over the role of the FDA in the era of precision medicine: When treatments target miniscule subsets of people with specific genetic illnesses and traits—can the agency do its job?

An op-ed published Oct. 24 in the Journal of the American Medical Association, coauthored by Aaron Kesselheim, an associate professor of medicine at Harvard Medical School and a member of the FDA’s Exondys 51 advisory committee, said the agency’s decision “provided a worrisome model for the next generation of molecularly targeted therapies.” Kesselheim’s concern was that more and more precision medicine treatments might get approved because they show positive effects in the body—like increasing protein levels or shrinking tumors—without actually making any measureable difference in patients’ health.

Traditionally, the FDA’s standard for evaluating drugs has been to ensure they’re safe and that they extend and/or improve patients’ lives. But since Congress signed off on so-called “accelerated approval” in 1992, instead of examining patients’ overall health, the FDA looks at “surrogate” measures—physiological indications that a drug is working. For example, in the case of a therapy for heart disease, the agency might require that it lower a patient’s cholesterol levels. A diabetes treatment might be evaluated on its ability to regulate blood sugar. Whether or not the patient experiences any improvement in symptoms after taking the drug doesn’t actually matter.

Sarepta submitted a study of 12 boys, all of whom took the drug—meaning the trial had no control group. The data showed little evidence of measurable improvement in participants gaining or retaining the ability to walk over time. So to illustrate Exondys 51’s efficacy, Sarepta pointed to the fact that three years after the trial ended, all of the boys had a protein called dystrophin in their muscle. DMD patients can’t make dystrophin themselves—and the trial participants, according to Sarepta’s data, had about 50% of the normal amount. But the FDA found flaws in its methods—and a re-analysis dropped the mean amount of dystrophin in the cohort to less than 1% of normal.

“It’s not just a surrogate, it’s just a whiff of a surrogate,” says Vinay Prasad, assistant professor of medicine at the Oregon Health and Sciences University, who has studied the approval of cancer drugs based on surrogate measure outcomes. “The primary reviewers”—the FDA’s advisory panel members—“don’t think there was a whiff there at all.”

In addition to studying the clinical data, the panel heard emotional pleas from families whose children are living with the disease. Ultimately, seven of the panel’s 13 members found that the change in dystrophin levels likely didn’t translate to patient improvement. Woodcock disagreed, writing in a July 14 memo that the protein’s presence was “reasonably likely to predict clinical benefit.”

Woodcock’s decision was appealed by Ellis Unger, the FDA’s director of the Office of Drug Evaluation, who sided with the advisory committee. But Robert Califf, who took over as head of the FDA early this year, backed Woodcock. The approval was granted with a request that Sarepta provide evidence that its drug works in a randomized clinical trial by the spring of 2021.

Shortly after it received the good news from the FDA, Sarepta tagged Exondys 51 with a cost of $300,000 per year. On a call this past Thursday morning with investors, Sarepta CEO Edward Kaye reported that the first patient was already receiving an infusion of the approved treatment.

“The FDA is supposed to make its decisions on black-and-white hard science and data,” says David Gortler, an ex-FDA senior medical officer who is now a regulatory expert with”Instead the FDA has succumbed to public pressure, trying to make everyone happy. Unfortunately, sometimes the answer in science is no.”

Another factor that probably influenced Woodcock’s decision: Exondys 51 probably won’t make any patients sicker. In several places in the FDA’s 125-plus page evaluation of the compound, the drug is deemed to be safe. That, says Darius Lakdawalla, a regulatory innovation expert at USC’s Schaeffer Center for Health Policy and Economics, allowed the agency to take a risk on a drug with uncertain effectiveness in an area where there was no treatment alternative. But, Lakdawalla, says, “I am very skeptical of regulatory agencies’ ability to weigh risks in the way patients’ would want them weighed.”

The Exondys 51 case follows a pattern set by the FDA’s approval of cancer drugs over the past decade or so, many of which also proved their worth through surrogate measures. Oncology is at the forefront of the precision medicine movement, the theory being that cancerous tumors with certain genetics might respond more favorably to a certain type of therapy. But thanks to the surrogate system, many therapies that been shown to simply decrease the size of tumors—without extending a patient’s life—have been approved.

A 2014 investigative report by the Milwaukee Journal Sentinel and MedPage Today found that of 26 cancer drugs that the FDA approved between 2004 and 2011 based on surrogate effects, 22 of them still had not been proven to extend life when the report came out. Often, companies that get approval for drugs through surrogates must later provide studies that show their treatments improve or extend patients’ lives—if they don’t, the FDA can fine them. But as of late 2014, the agency had never levied a single fine. (FDA could not provide information on whether fines were levied since late 2014 by press time.) The U.S. Government Accountability Office released a report late last year finding the agency was behind on reviewing more than half of the 1,400 studies it requested from 2008 to 2013 for drugs already on the market. In a response to the report, the FDA said it had identified and addressed issues causing the backlog, and by the end of the 2013/2014 fiscal year, only 13% of postmarket-study reviews were still delayed according to the FDA.

At a summit for the National Organization for Rare Diseases a month after Exondys 51 was approved, FDA Office of New Drugs Director John Jenkins noted the path to approval taken by Sarepta was “not a good model for other development programs,” suggesting the case was a one-off. He also made clear that randomized, controlled clinical trials are still the “fastest way” to determine a drug’s effectiveness.

“In this era of precision medicine, the agency expects that the issues associated with developing therapies for rare diseases will be magnified,” an FDA spokesperson wrote in an email to Quartz. “We firmly believe that patients with rare diseases deserve therapies evaluated with the same rigor as more common diseases.”

But not everyone agrees that amount of rigor is always applied, especially in cases like Exondys 51. That level of scrutiny isn’t always going to be possible when you’re dealing with very rare diseases or when you’re developing targeted therapies for small subsets of people, says Art Caplan, a bioethicist at NYU’s Langone Medical Center. The FDA’s approval system was designed to generate enough data to make people feel confident that drugs on the market are safe and effective. The precision medicine era means the agency will have to live with significantly more uncertainty—and that means elements like surrogate measures and the strength of a patient community are probably going to be a bigger part of the equation going forward.

“The good news is we have precision medicine,” says Caplan. “The bad news is we have a regulatory system that is designed for imprecise medicine.”