Medical Device Clinical Trials: What You Need To Know

Introduction

Medical device research is important in itself, as well as part of overall pharmaceutical research. It serves as a simple tool in the aid of diagnostic testing or as an alternative life-saving option in treating certain health conditions. Examples range from as simple as elastic bandages and tongue depressors to more complex pieces like blood screening instruments that reveal the presence of HIV or other diseases, or heart stents that save people’s lives. This paper will cover the differences between medical devices and pharmaceutical products, outline the clinical research process as compared to the pharmaceutical industry, and state some commonly used procedures for analyzing medical device data.

Differences between Medical Devices and Pharmaceutical Industries

There are some noted differences between medical devices and pharmaceutical industries in the literature. They are different in a number of ways as noted by Greg Campbell (2006). A medical device is anything that is not either a drug or a biologic product. Medical devices usually work physically, while pharmaceutical products usually work chemically or biologically. Medical devices can be therapeutic, diagnostic or something else, whereas pharmaceutical products are usually therapeutic. Medical devices are invented, while drugs are usually discovered.

Medical devices can be changed during clinical development and once on the market a newer, improved version may be in development. Thus, the life cycle of a medical device may only be as short as a couple of years. In contrast, drugs are usually on the market for many years. Medical devices are approved through the Premarket Approval (PMA) application process and a single confirmatory study is often sufficient for approval. In contrast, drugs are approved through the New Drug Application (NDA) process and drug development is characterized by Phases I through IV clinical trials. There are numerous medical device companies registered with the FDA, while in comparison there are only relatively few pharmaceutical companies. Medical device companies are usually small (the median size is less than 50 employees), whereas pharmaceutical companies tend to be large.

Approval Process for Devices

The FDA approval process for medical devices is different compared to drugs from pharmaceutical industries. Medical devices are submitted for approval to the Center for Devices and Radiologic Health (CDRH) or Center for Biologics Evaluation and Research (CBER) at the FDA, while drugs from pharmaceutical companies are submitted for approval to the Center for Drug Evaluation and Research (CDER) or CBER at the FDA. Not all devices need to go through controlled clinical trials to gain regulatory approval. If a device needs a confirmatory study to support a premarket approval (PMA), this does not rely on randomized concurrent control but on historical controls showing evidence that the device is “safe and effective.” This confirmatory study is usually enough to support a PMA application while pharmaceutical applications generally require two adequate, well-controlled, confirmatory clinical trials.

In an effort to ensure safety, medical device regulations are continuously under criticism. Different scientific groups are conducting their own research to propose a tougher approval process for a wide range of device that experience recalls because of failure to perform in thousands of patients causing several injuries. But the medical device industry and its allies argue that “more regulations slow innovation, harm patients and cost jobs.” While the FDA guarantees continuous awareness on the safety surrounding medical devices and keeping their regulations in check, the innovation and advancement of medical devices are moving so fast that regulatory changes and/or improvements would face challenges and protests.

Medical Device Classification

Medical devices are classified into Class I, II and III. In terms of regulatory requirement, Class I is controlled the least while Class III is controlled the most. Class I are generally simple devices that pose minimal risk to the users like enemas, bedpans and elastic bandages. Most Class I devices are exempt from Premarket Notifi cation 510(k). A 510(k) must establish that the device is significantly equivalent to one legally in commercial distribution in the United States before May 28, 1976 or to a device that has been determined by FDA to be substantially equivalent. Most Class II devices require Premarket Notification 510(k). Class II are devices that pose a moderate level of risk like intravenous administration sets, sutures and inflatable blood pressure cuffs. On the other hand, Class III devices are high-risk devices that may cause a significant risk of illness or injury, or devices found not significantly equivalent to Class I and II establish through the 510(k) process. Some examples are implantable pacemakers, blood vessel stents and breast implants. Most Class III devices require Premarket Approval (PMA) process which is more involved and comprises the submission of clinical data to support claims made for the device.

Medical Device Development

The development of a medical device follows a different route than of a drug. While clinical trials on drugs focus on dose response study, medical device clinical trials give attention to prototype development. Drug development follows an extensive Phase I, II, III and IV clinical trialing process to test for safety, efficacy and toxicity, whereas medical device has feasibility, pilot and pivotal study models. Some medical device research involves substantial bench and animal testing for reliability and biocompatibility like in implants, but there are no studies for toxicity on devices like the Phase I or animal studies required for pharmaceutical research. Pilot and feasibility studies on medical devices are considered first-in-man studies. Device development is iterative and designs may be refined or improved as device development progresses. While user feedback, adverse events or difficulties in deploying or delivering a device can all lead to changes to the device, second or third generation designs do not always require a new clinical trial. Bridging the new to the old design may require additional bench studies or small confirmatory post-market study.

Statistical Analysis

There are two types of studies in medical device research — reproducibility and clinical utility (Smoak, 2009). To prove the accuracy and precision of a device, a reproducibility study is conducted. For example, a qualitative diagnostic assay uses hit rates while a quantitative diagnostic assay utilizes coefficient of variation and precision analysis using linear mixed models (proc mixed in SAS®) to verify reproducibility. On the other hand, to demonstrate the real-life use of device in clinical practice, a clinical utility study is performed. For example, a diagnostic assay may be used to monitor subjects given either a treatment or placebo in a clinical trial. Typical analyses might include measures of sensitivity, specificity, positive predictive value and negative predictive value. Since diagnostic studies for medical devices can be much shorter in duration than pharmaceutical studies, SAS programmers may have a less time to program (Smoak 2008a).

• Measures of Diagnostic Accuracy.
The accuracy of any test is measured by comparing the results from a diagnostic test (positive or negative) to the true condition (presence or absence of disease) of the patient. The two basic measures are sensitivity and specificity. Sensitivity is the ability of a test to detect the disease status or condition when it is truly present, i.e., it is the probability of a positive test result given that the patient has the disease or condition of interest. Specificity is the ability of a test to exclude the condition or disease in patients who do not have the condition or the disease i.e., it is the probability of a negative test result given that the patient does not have the disease or condition of interest. In clinical practice, it is also essential to know how good the test is at predicting the true positives, i.e., the probability that the test will give the correct diagnosis, through their predictive values. The positive predictive value (PPV) is the probability that a patient has the disease or condition given that the test results are positive, and the negative predictive value (NPV) is the probability that a patient does not have the disease or condition given that the test results are indeed negative.

• Handling Missing Data.
One of the main challenges in clinical trial analysis is addressing how to handle missing data. Missing data may be caused by patients dropping out or withdrawing their consent, patients who are lost to follow up due to relocation or their living condition, or centers that are closing even before the study is completed. Since missing data can result in biased treatment comparisons and affect the interpretation of study results, it is important to run sensitivity analyses to evaluate the robustness of study results. In medical device clinical trials, one of the methods used to handle missing data is tipping-point analysis. A tipping-point analysis replaces the missing value with some values so that the resulting p-value of the hypothesis is equal to or larger than a prespecified significance level. These outcomes, called tipping points, may convey some questionably poor outcomes that may aid clinical reviewers in making a judgment about treatment effect in the study.

• Propensity Score Analysis.
Propensity score analysis is a versatile statistical method used mainly in observational studies for improving treatment comparison by adjusting for up to a relatively large number of potentially confounding covariates. A propensity score is the conditional probability of a patient receiving the active treatment rather than the control, given a collection of observed covariates. The purpose of a propensity score analysis is to attempt to simultaneously balance many covariates in the two treatment groups, in an effort to reduce bias. There has lately been an increased interest in applying this method to nonrandomized medical device clinical studies, which could present some statistical and regulatory issues in both the design and analysis of study results. A high degree of statistical expertise is required in handling issues like pre-specification of clinically relevant covariates to be measured, suitable patient populations, planning of sample size in the context of propensity score methodology, handling missing covariates in generating propensity scores, and assessing the success of the propensity score method by evaluating treatment group overlap in terms of the distributions of propensity scores. In general, devices go through continuous improvement in short intervals. It is not uncommon for a clinical trial to start with one device and end with an improved version of the device. Because of the knowledge accumulated over the years on some devices (e.g. pacemaker), it is possible to establish an objective performance criterion that is then imposed on a new device for the same purpose. In addition, the accumulated experience on the control has led many device companies to propose hierarchical models when designing and analyzing a device trial. The latter has led to the FDA guidance on the use of Bayesian statistics in medical device clinical trials (2010).

• Bayesian Clinical Trial.
Bayesian statistics is an approach for learning from evidence as it accumulates. While information from previous studies may serve as a supplemental idea in traditional (frequentist) statistical methods, it is not part of the formal analysis. On the contrary, the Bayesian approach uses this prior information and combines it with the current information about the data to conduct the analysis. The Bayesian idea takes into account the prior information and the trial results as part of a continual data fl ow where new and up to date inferences are done every time new data become available. An FDA document is available detailing the design and analysis of clinical trials for medical devices that use Bayesian statistical methods (Guidance for the Use of Bayesian Statistics in Medical Device Clinical Trials). Because of the mechanism and evolutionary development of medical devices, good prior information is often available. Through the Bayesian approach, this good information may be incorporated into the statistical analysis of a trial. In some situation, the prior information for a device may be a justification for a smaller-sized or shorter-duration pivotal trial. Additionally, the mechanism of action of a medical device is usually physical which results to local, not systemic, device effects that can sometimes be predictable from prior information.

• Adaptive Design.
Adaptive designs use accumulating data to determine how to adjust certain aspects of a trial according to a pre-specified plan, the most common being the notion of early stopping of a trial. For this possibility of early stopping, one or more interim analyses are performed prior to the final analysis. The plan is to assess if the accumulated evidence at the interim is sufficient to draw a suitable inference and make a sound decision. Another useful application of adaptive design is the ability to change sample size during the course of the clinical trial. The need for sample size re-estimation comes about because all sample size calculations make key assumptions about the primary study outcome. Adaptive trial designs can sometimes be easier to implement using Bayesian methods than frequentist methods. By adhering to the Likelihood Principle, a Bayesian approach can offer flexibility in the design and analysis of adaptive trials. From design to analysis of medical device data, varied technical expertise may be required. While most medical device data are acquired over a shorter duration of time and take less time for programmers to program, more sophisticated designs and analysis require highly technically trained statisticians and programmers especially when handling adaptive design and Bayesian clinical trials. These designs require extensive pre-planning and model-building from the prior information to mathematical modeling and combining the information being gathered.

Conclusion
The medical device industry is inevitably growing and becoming more important. Its clinical research is very essential in assessing the safety and effectiveness of numerous medical devices in the market or in the development process. It is also a very important element in pharmaceutical research, like devices used to deliver drugs or diagnostic imaging to monitor therapies. Medical devices are different from pharmaceutical products in terms of FDA approval process, pace or duration of study, and the types of studies and corresponding statistical analysis being employed.

REFERENCES
Campbell G. “The Role of Statistics in Medical Devices – The Contrast with Pharmaceuticals.” Biopharmaceutical Report 14:1-8, 2006.

Mandrekar JN, Mandrekar SJ. “Statistical Methods in Diagnostic Medicine using SAS® Software.” Proceedings of the SAS® Users Group International, April 2005. < http://www2.sas.com/proceedings/sugi30/211-30.pdf >

Meier B. “Study of Medical Device Rules Is Attacked, Unseen” The New York Times, July 2011.

“Medical Devices Industry Assessment”, Medical Market Fact Book 2008 < http://www.trade.gov/td/health/Medical%20Device%20Industry%20 Assessment%20FINAL%20II%203-24-10.pdf >

Smoak C. “Medical Device and Diagnostic Industry 101.” 2010. < http://www.pharmasug.org/cd/papers/IB/IB01.pdf >

Yan X, et.al “Missing data handling methods in medical device clinical trials.” Journal of Biopharmaceutical Statistics, Nov 2009; 19(6):1085-98.

Yu Shu, et.al “Tipping-point Analysis in Medical Device Clinical Trials” 4th Annual FDA/MTLI Medical Device and IVD Statistics Workshop, April 2011.

< http://www.advamedmtli.org/download/fi le/Shu_Tipping-point%20Analysis%20 Day1.pdf >

Yue LQ. “Statistical and regulatory issues with the application of propensity score analysis to nonrandomized medical device clinical studies.” Journal of Biopharmaceutical Statistics, 2007; 17(1):1-13