Medical Device Link.

REGULATIONS & STANDARDS

Donald M. Powers, PhD, is president and principal consultant of Powers Consulting Group (Pittsford, NY) and is a member of IVD Technology’s editorial advisory board. He can be reached at powers@frontiernet.net.

The four previous articles in this series explored how IVD manufacturers can apply the risk management principles of ISO 14971, and integrate them into a quality management system.^1–5 This final article will examine the relationship of risk management to the vigilance activities that regulatory authorities require in order to protect the public health. This article will also highlight gaps that are sometimes found in risk management programs, and discuss the changes coming in the next edition of ISO 14971, which is scheduled to be published in early 2007.

Vigilance, Corrections, and Removals

When evaluating device malfunctions that may need to be reported to regulatory authorities, health hazard evaluations should be consistent with the risk estimates in the product risk analysis. While this may seem like common sense, many IVD companies evaluate such malfunctions one at a time, based on consultations with physicians about whether any serious harm could have resulted. The numbers and types of medical device reporting (MDR) filed by IVD manufacturers vary considerably.⁶

Determining whether device corrections or removals reduce a significant health risk (and are therefore reportable to FDA) is done the same way. However, evaluations that are performed during the heat of a recall are inevitably rushed and may not be objective. When decisions about reportable malfunctions are made without reference to the severity and probability levels documented in the risk management file, some conclusions about the degree of risk will be inconsistent. A decision not to report a malfunction would be untenable if the product risk assessment concluded that serious injury is a likely consequence. Regulatory reporting processes should be coordinated with risk management in a closed-loop quality management system.

If the patient consequences were properly considered during the product risk assessments, aligning postmarket health hazard evaluations is straightforward. But in order to avoid confusion, the terminology used in these two processes should be harmonized. For example, the scales used in risk analysis for severity of harm must clearly identify serious injury as defined in the MDR regulation.

Harmonizing the probability of occurrence estimates is more difficult, since the probability that a malfunction will recur does not enter into MDR decision making. The MDR regulation presumes that a failure that occurred once will occur again, and is only concerned with whether the malfunction is likely to lead to serious injury or death. On the other hand, risk analysis is based on the combined probabilities that a malfunction will occur and lead to harm. Product risk analysis documentation should include both probabilities: the probability that a malfunction will occur and the probability that harm will result from the malfunction.

Another reason to harmonize the terminology is that FDA requires IVD manufacturers to file MDRs for malfunctions when the likelihood of serious harm is not remote. While risk analysis probability scales commonly include the term remote, manufacturers do not always define it the same way as the MDR regulation guidance.⁷ Conceptually, a remote probability means that no serious injury is likely to occur from the malfunction being analyzed.

Figure 1. (click to enlarge) Risk Management and the MDR/Vigilance System.

The benefit of tying MDR evaluation criteria to the risk management process is that it allows IVD manufacturers to streamline their MDR decision making (see Figure 1). For example, if product risk assessments are kept up-to-date, most reportable device malfunctions will be predetermined. To facilitate MDR evaluations further, screening criteria can be developed from the risk analyses and used by the complaint handling unit to rule out the clearly nonreportable events. Furthermore, complaints involving incorrect test results can be classified based on the magnitude of the error. If an error does not reach a predetermined threshold, it is considered nonreportable. Such criteria must be objectively based on the medical uses of the test results, as determined by qualified medical professionals.

Figure 2. (click to enlarge) Error grid for blood glucose testing.

Error grids are an elegant way to classify incorrect IVD results according to the risk to patients.⁸ An example of a glucose error grid is shown in Figure 2. The consensus methodology developed for glucose error grids can be adapted to establish risk classification criteria for many quantitative IVD assays. In the blood glucose example, when the actual glucose concentration is less than 40 mg/dL, glucose readings of up to 60 mg/dL are considered clinically benign (zone B in Figure 2). As long as there are no reports of actual injury attributed to glucose errors in this zone, such a complaint would require no further evaluations as a potential MDR. On the other hand, the error grid indicates that glucose readings greater than 160 mg/dL would be hazardous if the actual glucose concentration is less than 40 mg/dL (zone E). Since hypoglycemic patients could suffer serious consequences if given insulin treatments, such complaints would be reportable device malfunctions.

If the product risk assessments contain sufficient details about the clinical uses of the test results, objective MDR decision criteria can be established. Then, the time and resources saved from agonizing over whether a malfunction is reportable can be focused on investigating and correcting the underlying causes of the complaint.

Risk-to-health decisions are required whenever product corrections or removals are initiated. Official FDA recall classifications are based on health hazard evaluations, which are in effect risk assessments. IVD manufacturers must perform their own evaluations in order to determine if their actions reduced significant risks to health so they can comply with the notification requirements of the corrections and removals regulation. At the same time, this regulation also requires that manufacturers must justify and document any decisions not to report a field correction or product removal. Many IVD companies document their decisions by using the same template that FDA uses for health hazard evaluations.⁹ If the necessary reporting criteria are developed as part of the risk management process, as they should be, such decisions can be made quickly and objectively with available information, and without the need to engage external medical experts each time a field action is taken.

Common Risk Management Gaps

Risk management requirements for IVDs have been in place in the United States for the past 10 years, and the IVD Directive became official in the European Union in 2003. However, compliance audits and discussions with industry colleagues have indicated that many IVD manufacturers are still struggling to integrate risk management into their quality management systems. While this situation is not unique to the IVD industry, the indirect nature of the risks to patients add a challenging dimension. Manufacturers evaluating their risk management programs against ISO 14971 should be mindful of the following common oversights.

Poor documentation. When records fail to support an objective risk analysis, insufficient time, insufficient resources, and inadequate training on the risk analysis techniques are usually blamed. Even if a dedicated team performed a thorough analysis, its conclusions will be unconvincing without evidence that it considered pertinent data and justified its decisions.

Incomplete risk analysis. The lack of a systematic approach results in failure to identify potential hazards. A team that performs risk analyses too hastily may ignore significant failure modes or may neglect to consider all foreseeable uses of the product and its test results. High recall rates tend to go hand-in-hand with inadequate risk analyses.

Unjustified assumptions. The clinical significance of incorrect test results and the medical requirements for performance characteristics require input from qualified medical experts. The burden is on the IVD manufacturer to know how its assays will be used and understand the consequences when they fail to perform as intended.

Overlooked risks in normal use. No qualitative assay achieves 100% diagnostic sensitivity and specificity, and no quantitative assay demonstrates perfect correspondence to reference method values. Analytical error is a characteristic of IVD assays that creates inherent risks because a percentage of laboratory results will be clinically incorrect even though the assay is meeting all of its performance claims. Such inherent risks must be evaluated for acceptability during the design phase, and monitored during the postlaunch phase. The performance characteristics of established state-of-the art assays play a significant role in setting the acceptability criteria.

Stale risk assessments. Risk management analysis documentation may not be up to date due to one of several reasons: MDR experience was not incorporated, customer complaint data failed to support risk analysis conclusions, and design/process changes were not evaluated for their impact on the original risk estimates.

Out-of-date risk assessments can occur when the departments responsible for different risk management activities do not communicate, especially if their quality system procedures are poorly linked. Risk assessment outputs should be captured in a living document with future uses in mind. Not documenting risk assessments in a way that facilitates their use throughout the product life cycle creates problems in the future. Risk assessments must be reviewed and updated regularly with input from other processes, such as complaint investigations, change controls, and corrective action/preventive action (CAPA).

Focus on the wrong risks. Process failure modes and effects analysis (FMEA) are often performed for production optimization, ignoring patient safety risks. This is usually symptomatic of a failure to integrate process development into the design control process. The process FMEA is then performed independent of the IVD company’s design control and risk management programs. Process failure modes are consequently not linked to the product’s intended uses and safety characteristics or potential patient harm, resulting in an inadequate risk assessment.

Management insufficiently engaged in risk management. Often no formal policy exists on how an IVD company determines risk acceptability, or the policy lacks criteria for what level of risk is acceptable. Risk decisions are sometimes delegated by lower level engineering staff without even requiring management approvals. Risk management is a fundamental responsibility of a company’s management.

The Future of Risk Management

This series will close with an update on ISO 14971. The first edition has been accepted as the definitive worldwide risk management standard for medical devices. The risk management principles of ISO 14971 have been incorporated into other medical device standards, and is acknowledged as an efficient and effective method of harmonizing risk management concepts for medical devices. The Global Harmonization Task Force (GHTF) is promoting the integration of the risk management principles of ISO 14971 into quality management systems.¹⁰

The standard is presently undergoing revision as required periodically for ISO and IEC standards. In February 2006, the Draft International Standard was approved by a vote of 18 to 3. The issues raised by the three dissenting votes were addressed at a meeting of Joint Working Group 1 (JWG1) of ISO/TC210 and IEC/SC62 in April 2006. A Final Draft International Standard is being prepared for a vote by the ISO membership. The second edition is expected to be published in early 2007.

Because ISO 14971 has already become so widely accepted, JWG1 deliberately avoided making any substantive changes to its requirements. While some improvements were made to ease the application of the standard, the major change is an expansion of the extensive and comprehensive annexes. IVD manufacturers should find the annexes of the second edition very helpful. For example, one annex prepared by ISO/TC212 contains a wealth of risk management guidance written specifically for IVD manufacturers. Other annexes contain informative discussions of risk concepts and detailed guidance on risk management techniques.

Conclusion

One element that is still missing in the systematic management of risks to patients is the participation of clinical laboratories. IVD manufacturers can only do so much to prevent incorrect test results, as testing is under the control of the laboratories, and the majority of testing failures originate in the pre-analytical phase. Laboratories can use the concepts described in ISO 14971 to incorporate risk management into their quality systems, and there is some indication the idea is catching on. In April 2006, the theme of the annual leadership conference of the Clinical and Laboratory Standards Institute (CLSI; Wayne, PA) was “Creating a Cultue of Patient Safety Through Risk Management,” and an accompanying workshop on “Risk Management Tools for Improved Patient Safey” was well attended by the laboratory community.¹¹ In time, a seamless risk management framework may emerge, with clearly defined roles for manufacturers and laboratories as partners.

---