Despite the regulatory approval of new antiepileptic drugs (AED) over the past decade and the expansion of indications deemed suitable for epilepsy surgery, individuals with intractable seizures – roughly one in three epilepsy patients – still face a dearth of treatment options. That void is all the more frustrating because epilepsy remains a debilitating condition for reasons beyond the clinical manifestations, often resulting in chronic unemployment or underemployment, social isolation and poor socioeconomic status.

It's no surprise, then, that the prospect of a new treatment for people with medically refractory epilepsy would be greeted with great enthusiasm. As an adjunct to AEDs, the responsive neurostimulator system, or RNS, under development by Mountain View, Calif.-based NeuroPace, is designed to detect abnormal electrical brain activity and deliver electrical stimulation to regulate brain activity before a seizure develops. The RNS could become the first electrical stimulation device approved by the FDA in epilepsy since vagus nerve stimulation was green-lighted in 1997.

But the results of the Feb. 22 FDA adcom meeting evaluating the NeuroPace RNS raise troubling questions about regulatory decisions in epilepsy care. As reported in Medical Device Daily, the FDA briefing documents submitted for the Neurological Devices Panel of the Medical Devices Advisory Committee meeting indicated that a pre-specified statistical analysis plan of NeuroPace's prospective, randomized, double-blind, sham-stimulation-controlled pivotal study called for the use of a generalized estimating equation (GEE) model and required that superiority be demonstrated based on a reduction in the frequency of total disabling seizures during the blinded evaluation periods (BEP).

However, neither the study protocol nor the statistical analysis plan “explicitly stated” which of two methods would be used to estimate standard error, producing what the FDA called “distinctly different p-values” for the primary efficacy endpoint. The agency and sponsor concurred that an alternative model was needed, agreeing to use the Poisson regression model. Although the FDA recommended that monthly seizure counts be used, NeuroPace chose the daily model, with the expectation that many seizure-free days would occur in the treatment group and there would not be a clinically meaningful way to group data.

That premise seemed reasonable, since seizure activity can vary from day to day. However, the observed variability in the study exceeded the expected variability in daily seizure counts following a Poisson distribution. NeuroPace then made ad hoc modifications, reverting to monthly seizure counts along with the use of a negative binomial distribution to analyze the returns. According to the FDA, NeuroPace had to add an adjustment for clinical co-variates along with the other modifications to achieve statistical significance for the primary efficacy endpoint.

With these modifications, the pivotal study showed a statistically significant 37.9 percent reduction in seizure frequency in the treatment group compared to a 17.3 percent reduction in the control group during the three-month BEP. Long-term results showed median seizure frequency reductions of 44 percent and 53 percent, respectively, at one and two years post-implant.

Statistical Qualms Bubble Up

In its briefing docs, the FDA argued that some alternative post-hoc GEE models did not achieve statistical significance, none of the pre-specified secondary endpoints achieved statistical significance and no observed data analyses achieved statistical significance. Moreover, response to the device varied by the number of seizures seen at baseline, and findings from the treatment arm may have benefited from two patients on the sham arm with a different response pattern than others in the trial.

Despite these statistical qualms, 12 of 13 adcom members agreed there was “reasonable assurance” that the NeuroPace RNS system is effective for patients meeting the criteria for the proposed indication: adults with partial onset seizures from no more than two foci who are refractory to two or more AEDs. One panelist – a statistician – abstained. The panel voted unanimously that the system is safe, and 11 members concurred that the RNS benefits outweighed the risks, with two abstentions.

I understand the genuine desire to offer hope to epilepsy patients with uncontrolled seizures. My husband developed epilepsy at age 15 a year after a serious head injury, and he is not seizure-free after 40 years on various combinations of AEDs and a left temporal lobectomy.

But should the FDA permit statistical manipulation in epilepsy that would likely raise eyebrows in other indications? At BioWorld Today, I write about the development of drug therapies designed to treat individuals with cancer, hemophilia, multiple sclerosis, Alzheimer’s disease and many other poorly treated diseases. I understand the desire to bring new treatment options to patients with agonizing conditions. I really do.

I also see countless drugs rejected by the FDA when efficacy is not assured, even when safety is unquestioned. And, compared to medical management, the implantation of a deep brain stimulator is not inherently safe. Up to one in five patients could be expected to have a serious adverse event related to implantation and related medical procedures, according to the FDA’s own risk/benefit analysis of the RNS. Shouldn’t the burden of proof for such a device be even higher than that of a drug?

Above All, Demonstrate Efficacy

With respect to epilepsy, quality of life implications often are cited as a compelling reason for drug or device approval, with patients seemingly willing to accept a higher risk threshold in return for the prospect of improved seizure control. In my view, that premise should be predicated on demonstrated efficacy. In the real world, physicians are quick to suggest new treatments, and patients often are desperate. Few have the benefit of advocates with neurological expertise who can provide an unvarnished appraisal of a newly approved therapy. I wake up every morning with someone who has run the epilepsy treadmill, trying therapies touted for their efficacy that proved futile, and worse.

The NeuroPace RNS may, indeed, help some epilepsy patients, which would be a terrific outcome for the developer and the epilepsy community, alike. However, that claim seems far from proven, which should give the FDA pause in approving its usage without additional study. Individuals with epilepsy deserve treatments that offer true hope, not outcomes derived on the basis of statistical tinkering.

Editor’s note: To read more about Marie and Chuck Powers’ experiences with epilepsy and an unusual condition that surfaced after a temporal lobectomy ‑ prosopagnosia ‑ visit her personal blog, About Face.