Medical Device Link.

Originally Published IVD Technology July 2003

REGULATIONS & STANDARDS

Donald M. Powers, PhD, is president of Powers Consulting Services (Rochester, NY), an independent IVD regulatory and quality consulting firm. He can be reached at powers@frontiernet.net.

Risk management is the overarching essential requirement of the IVD Directive.¹ The directive puts considerable emphasis on an IVD manufacturer’s obligation to minimize risks to the patient, the public health, and the environment. As an indication of the importance European regulators place on avoiding unnecessary risk, the term “risk” appears 34 times in the text of the directive, compared with only once in the U.S. quality system regulation (QSR).²

U.S. regulators nonetheless share their European counterparts’ concern for minimizing risk. The preamble to the QSR clearly indicates that manufacturers are expected to establish a systematic risk management process. In fact, the only practical differences between the two regulatory schemes are that the QSR does not require risk analysis for Class I devices or unchanged existing products, and the IVD Directive does not exempt any products. 

Although this article addresses risk management as a regulatory requirement, compliance is not the sole reason to implement a risk management program. A systematic approach to minimizing risk in IVDs not only makes good business sense but also conforms to the ethical norms of society. A recent article describes the many benefits of integrating risk management into the design, development, and manufacture of medical devices.³

Basic Requirements

The IVD Directive mandates that risks associated with the use of an IVD must be acceptable when weighed against the benefits to the patient, and must be compatible with a high level of protection of health and safety. IVDs must be designed and manufactured to reduce risk as far as practicable from more than 20 types of hazards mentioned in Annex I of the essential requirements.

The principles that IVD manufacturers must apply in selecting appropriate risk-reduction solutions are specified in order of preference. Manufacturers must:

• First, eliminate or reduce risks as far as possible through inherently safe design and construction.
• Second, implement and verify adequate protective measures for risks that cannot be eliminated.
• Last, inform users of residual risks if adequate protection measures are not practicable.

The harmonized standard that supports this essential requirement (ISO 14971) requires company management to set risk management policies, provide qualified resources, ensure adequate training, and be actively involved in risk management decisions. Management’s responsibilities cannot be delegated. Manufacturers must formulate a risk management plan that includes criteria for acceptable risk, document all risk decisions in a living risk management file, and systematically review postproduction experience.⁴ In case a potentially harmful product failure occurs, manufacturers must also have effective recall and corrective-action processes in place to reduce or eliminate possible risks. 

Types of Risk 

The framers of the IVD Directive recognized that IVDs do not pose serious risks to patients. They acknowledged that IVDs are generally used by competent professionals and that physicians routinely corroborate most laboratory results by other means and question those that do not fit the clinical impression. However, the framers also recognized that some assays form the primary basis for diagnostic and therapeutic decisions, and that erroneous results from these could have life-threatening consequences. To determine the level of regulation necessary to protect the public, the directive classified IVDs according to risk. Significant risk is presumed if accurate test results are considered essential for effective medical practice and if failure to produce accurate results could cause serious harm to a patient’s health. 

For these assays, including all self-testing devices, European Union (EU) regulators mandated intervention by a notified body to verify that the manufacturers performed the risk management activities satisfactorily. 

The remaining IVDs are considered lower risk, but not risk-free. A signed declararion of conformity and a CE mark on these products are needed to sell it in Europe. 

Risk Standards

Figure 1. The risk management process. Source: ISO 14971:2001. Click to enlarge.

To justify allowing manufacturers to conduct their own conformity assessment, EU regulators mandated a voluntary standard so manufaturers could evaluate risk consistently and objectively and lower risks to an acceptable, or at least tolerable, level. Although manufacturers are solely responsible for deciding the acceptability of most IVDs, the criteria have to be supported by a reasonable justification.

ISO 14971 describes a complete risk management program and, as a European harmonized standard, it conveys a presumption of conformity with the essential requirements.

ISO 14971 establishes a framework in which manufacturers can apply their experience, insight, and judgment in a systematic way to manage the risks related to the use of their products. This standard relies on procedures already used in the medical device industry. According to the standard, manufacturers must identify hazards associated with the device, estimate and evaluate the risks associated with the hazards, control any residual risks, and monitor the effectiveness of the controls. The standard does not specify acceptability criteria for IVD hazards; the manufacturer must determine what is acceptable. To preserve objectivity, manufacturers must establish the acceptability criteria before risks are evaluated. 

Risk analysis, risk evaluation, and risk assessment are often used interchangeably, which can make communication difficult. The process flow diagram from ISO 14971 helps to differentiate these terms (see Figure 1). 

Harm is physical injury or damage resulting from a hazard, not only to health but also to property and the environment. The concept of risk combines the probability of harm occurring with the severity of that harm. Risk analysis is the systematic use of available information to identify hazards and estimate the risks. Risk evaluation involves judging the acceptability of a risk based on the values of society. Risk assessment is the overall process, which analyzes and evaluates risks. Together with risk control and monitoring the effectiveness of risk controls through the product life cycle, these processes constitute risk management. 

Risk Analysis

The risk management process starts by defining the product’s intended use and identifying foreseeable misuses and off-label uses. Although most IVDs would not directly cause harm to patients, a product failure could trigger a chain of events that ultimately results in patient harm. For example, an IVD failure can cause the clinical laboratory to report inaccurate results, which can contribute to a erroneous medical decision that leads to injury or death of the patient (see Figure 2). Such hazards are considered indirect; erroneous results have to be accepted by the laboratory and physician in order for harm to occur. This significantly reduces the probability of harm for most analytes. 

While product failures predominantly affect the analytical process in the clinical laboratory, they can also contribute to pre- and postanalytical errors (e.g., inaccurate or incomplete sample preparation instructions, misidentification of reported results). IVD manufacturers must thoroughly analyze all such potential hazards for their causes and effects. Manufacturers also must not overlook hazards to laboratory workers from using and disposing of radioactive, infectious, or toxic reagents, hazards to service personnel from contaminated devices (i.e., biohazards), and hazards to the environment. 

In addition, IVD manufacturers must consider any known and foreseeable hazards that might arise while shipping, storing, using, and disposing of their devices, as well as any foreseeable sequences of events that could result in a hazardous situation, including off-label uses.

It may not seem obvious, but hazards can occur in normal use because no analytical method has perfect accuracy. Qualitative assays have inherent false- positive and false-negative rates, and quantitative assays may exceed the accuracy limits defined by clinical utility a predictable percentage of the time (e.g., 5%). These inaccurate results must be evaluated during the design phase as potential hazards to verify the risk to patients is acceptable.

Some examples of failure modes that are common to many IVDs are the following: calibrator inaccuracy; lot inhomogeneity; lot-to-lot inconsistency; interference; carryover effects; reagent degradation; inadequate instructions for collecting, preparing, storing, and analyzing clinical specimens; and lack of robustness of the assay design.

Instrument-related hazards to laboratory workers and service personnel, such as electric shocks, unguarded moving parts, pipetting needles containing potentially infected serum, containers of contaminated waste solutions, etc., must also be evaluated.

A number of memory joggers are available to help identify potential hazards. ISO 14971 contains an annex with examples of possible failure modes, and NCCLS published a checklist applicable to unit-use IVDs.5 IVD manufacturers can use these references to create lists that specifically apply to their devices.

The expertise and experience to perform a satisfactory risk analysis rarely reside in one person. The usual industry practice is to form a cross-functional team representing the necessary departments and disciplines. Participants in the risk analysis must be carefully selected based on their ability to work as a team, and membership must be balanced to avoid having any one perspective dominate the discussions. 

Risk Evaluation 

Once a list of product-specific hazards has been developed, IVD manufacturers must evaluate each one against the criteria for acceptable risk defined in the risk management plan. Manufacturers can obtain information and data for estimating risks from many sources, including expert opinions. Manufacturers commonly assign numerical values to each hazard and decide whether the estimated risk is low enough to be acceptable. This step is usually the most difficult for manufacturers because it requires knowledge of medical decision-making that companies may not have. While risk assessment teams generally know the product capabilities and manufacturing processes, they may not fully understand how their products are used in medical practice. 

For example, scientists and engineers who develop products often stress the importance of test results in medical decision-making, so they tend to overestimate risks and potential harm. At the same time, technical personnel who support products may view performance problems as routine, therefore underestimating their medical significance. For these reasons, it is imperative for IVD manufacturers to ensure that unbiased medical expertise is available to the risk assessment teams; if not as an active participant, then at least in the beginning to start off on the right foot and at the end as a reality check. These medical consultants should also have a good understanding of laboratory medicine and an awareness of the inherent uncertainty of laboratory results. The error grid approach is a useful way to capture medical input and document the effects of analytical errors on clinical decision-making.⁶

Risk Control

Figure 1. The risk management process. Source: ISO 14971:2001. Click to enlarge.

Risks are commonly classified based on a combination of the severity of potential harm and the probability of it occurring. Risks can range from broadly acceptable to intolerable. Risks at the broadly acceptable level require no further risk controls. However, other risks must be reduced to “as low as reasonably practicable” (ALARP) even if they meet a manufacturer’s acceptability criteria. This can be done by reducing the severity of the potential harm, lowering the probability that harm could occur, or both. 

Some IVD manufacturers provide specific quality control instructions intended to detect conditions that could produce erroneous results. ISO TC212 has developed a standard, ISO 15198, to help manufacturers validate quality control procedures as a risk control.⁷

Two levels of verification are required for the risk controls called for by the risk management plan. Manufacturers must not only demonstrate that the control measures have been implemented, but also provide objective evidence that they are effective in reducing the risk.

Risk/Benefit Analysis

Practicable means solutions to control risk are technically available and economically feasible. However, even if the risks are reduced to ALARP, in some cases they still do not meet the acceptability of risk criteria. In these situations, IVD manufacturers must conduct a risk/benefit analysis to determine if the medical benefits outweigh the residual risk. If the evidence supports this conclusion, then the device can be marketed for limited intended uses, provided an explanation of the residual risk is included in the product labeling. Otherwise, the risk remains unacceptable, and the device is not eligible for a CE mark. 

Postproduction Monitoring

Postproduction monitoring is an integral part of the risk management program. IVD manufacturers must ensure that the risk assessment remains valid throughout the product life cycle. Manufacturers must systematically evaluate complaints, external quality assurance system performance, scientific publications, other customer feedback, and their own internal quality data to detect previously unrecognized hazards as well as changes in circumstances that have resulted in known risks becoming unacceptable.

Many IVD companies fail to connect their vigilance systems to their risk assessments. These companies end up making independent and sometimes inconsistent decisions when determining if a health hazard exists. This may be hard to explain when a competent authority reviews the technical dossier. Decisions about harm severity and risk made during the design and development stages should provide the objective basis for complaint classifications, vigilance reports, safety alerts, and recall decisions.

Existing Products 

The risk management approach described in ISO 14971 is easily integrated into new-product development processes. However, products that have been on the market for 20–30 years or more present a special challenge. Even though risk analysis was unknown to the IVD industry at that time and pertinent development data may no longer exist, the IVD Directive still requires a documented risk analysis for every product. 

While some might argue the absence of adverse events demonstrates acceptable risk, objective acceptability decisions must be based on scientific evidence. On the other hand, a substantial, well-documented history of performance should count as much as thought experiments conducted during the design phase. The key is to analyze the data properly to draw a scientifically valid conclusion. 

The analysis should start with the intended use. Unless assay results are the basis for critical medical decisions, erroneous results are not likely to cause serious harm, and therefore the hazard potential is inherently low. An analysis of complaints that shows a low failure rate and no adverse events during a sufficient number of representative manufacturing events should be sufficient to convince authorities that the risk is acceptable.

If an erroneous result could contribute to serious harm, however, a detailed risk management plan including a thorough hazard analysis would be expected. Risk controls must be commensurate with the degree of harm that could result. Identification of essential control points in the manufacturing process helps to determine whether existing controls are adequate to prevent hazardous failures or whether implementation of additional risk controls is necessary.⁸

Risk Communication 

Communication of residual risks is an essential, but often neglected, part of the risk management plan. IVD manufacturers should develop a risk communication policy to ensure that essential information is communicated effectively. 

For residual risks that are judged acceptable and cannot reasonably be lowered, IVD manufacturers are obligated to inform users (laboratory directors for professional-use products, patients for self-testing devices). For laboratory assays, such information is traditionally communicated in the product labeling, and laboratory directors assume the responsibility for informing and educating physicians served by the laboratory. Even though the IVD Directive lists labeling as the least desirable risk control, sometimes it is the only practicable solution.

Self-testing devices present an additional communication challenge because they are used by laypersons and there is no intermediary between the test result and the patient, who may act on the result without seeking professional advice. In the end, the manufacturers are obligated to verify that the labeling, including the method of communication, is effective in reducing the risk.

Conclusion

The IVD Directive’s explicit emphasis on risk management has elevated its role as a necessary life-cycle activity for IVD devices. ISO 14971 gives IVD manufacturers a systematic risk management process that, when followed, conveys a presumption that the manufacturer has complied with the directive’s risk management requirement. Conscientious risk management offers benefits that extend well beyond regulatory compliance. The result should be safer, more-effective IVD products for laboratories and patients. 

References

1. “Directive 98/79/EC of the European Parliament and of the Council of 27 October 1998 on In Vitro Diagnostic Medical Devices,” Official Journal of the European Communities L331 (1998): 1–37. 

2. “Medical Devices; Current Good Manufacturing Practice (CGMP) Final Rule; Quality System Regulation,” in Federal Register 
61(195):52602–52662, October 7, 1996 (codified in 21 C.F.R. parts 808, 812, 820). 

3. A Snow, “Integrating Risk Management into the Design and Development Process,” Medical Device & Diagnostic Industry 23, no. 3 (2001): 99–111. 

4. Medical Devices: Application of Risk Management to Medical Devices, ISO 14971 (Geneva: International Organization for Standardization).

5. “Quality Management for Unit-Use Testing,” NCCLS Guideline EP18-A, (Wayne, PA: NCCLS, 2003). 

6. JL Parkes et al., “A New Consensus Error Grid to Evaluate the Clinical Significance of Inaccuracies in the Measurement of Blood Glucose,” Diabetes Care 23 (2000): 1143–1148.

7. In Vitro Diagnostic Medical Devices—Validation of User Quality Control Procedures by the Manufacturer, ISO/FDIS 15198 (Geneva: International Organization for Standardization, 2003). 

8. DM Powers, “Validation and Other Process Controls: Prioritizing Remediation Activities for On-Market Products,” IVD Technology 8, no. 2 (2002): 22–26. 

Copyright ©2003 IVD Technology