Measuring Uncertainty in Medical Device Trials with Adaptive Designs

Biostatistics is the study of uncertainty and determining how to measure it and how to react based on the results. Medical Device trials face the following uncertainties:

  • Safety Problems
  • Unexpected treatment effects/safety issues
  • High variance
  • Effects in secondary endpoints/subpopulation
  • Reducing uncertainty in the planning phases of device development can eventually reduce timelines, and inevitably, costs.

Adaptive Trial Design is one way to achieve this.

What makes device development uncertainty different from drug development? While both test for proof of safety and efficacy, statistical methods in device studies test for estimation of effects while drug studies focus on hypothesis testing.

An adaptive design clinical study is defined as a study that includes a prospectively planned opportunity for modification of one or more specified aspects of the study design and hypothesis based on analysis of data (usually interim data) from study subjects (according FDA guidelines).

Adaptive Designs modify aspects of the study without undermining the validity and integrity of the trial.

  • Why do we adapt medical device trials?
  • We can’t always rely on assumptions in the planning phase
  • To make mid-course corrections for trials
  • To include fewer patients in a trial less likely to succeed
  • To reduce the development phase timelines
  • To increase the chance that patients will receive an effective treatment
  • What do we adapt in medical device trials?
  • Early termination
  • Sample size re-assessment
  • Treatment allocation ratios
  • Treatment arms (dropping, adding arms)
  • Hypotheses (non-inferiority vs superiority)
  • Test statistics
  • Population (inclusion/exclusion criteria)
  • Combine trials/treatment phases (seamless designs)

FDA vs EMA Guidelines for Measuring Uncertainty

The FDA lays out guidelines for situations which raise concerns and should be avoided during the planning phase including: multiplicity and controlling Type I error, choice of analysis made after unblinded data of interim are available, changes in the primary endpoint and operational bias.

The FDA also highlights guidelines for adaptive design study protocols:

  • Description of relevant information on study drug
  • Description of all possible adaptations
  • Summary of each adaptation with respect to statistical issues
  • Computer simulations
  • Details of analytic derivations
  • Details on blinding

The EMEA has considerations and requirements for design modifications such as minimal requirements for Type I error control, estimation of treatment effects with CI, ensuring that different stages can be combined and involvement with the Sponsor. With regards to design modification, the EMEA recommends keeping sample size reassessment blinded whenever possible and to avoid modification of the primary endpoint.

In addition, the EMEA recommends:

  • Imbalanced randomization over stopping arms
  • Randomization Ratio
  • Switching between non-inferiority and superiority trials should be hierarchical

Lastly, what are the implications of adaptive design on device trial project management?

In terms of statistical planning, the statistician will be needed for a longer period of time and should be regularly involved in the communication with trial personnel. A very detailed description of methods and adaptations have to be in the protocol along with setting up an independent Data Monitoring Committee. In order to have fast access to data and efficient data management, consider EDC solutions, a central database, central randomization, prompt data entry and efficient data cleaning processes.

Our expert statisticians have helped numerous companies implement adaptive trial design for both device and drug studies.