Medical Device Link.

REGULATIONS & STANDARDS

Donald M. Powers, PhD, is president and principal consultant of Powers Consulting Group (Pittsford, NY) and is a member of IVD Technology’s editorial advisory board. He can be reached at powers@frontiernet.net.

The first three articles in this series discussed the significance of risks from IVD devices, the importance of risk management planning, and the advantages of the risk assessment and control principles of ISO 14971.^1–3 This fourth article will look at the monitoring phase of the risk management process of ISO 14971, which continues as long as a product is in use.⁴ It will also explore ways to integrate risk management activities into a quality management system.

Monitoring Risks throughout the Product Life Cycle

A new product launch begins the most challenging phase of risk management for IVD companies. Up to this point, product risks are only theoretical, but when devices are in use the manufacturer learns whether all the significant hazards were taken into account and whether the risk estimates were correct.

This is also the phase during which conformance to ISO 14971 tends to break down. In general, IVD manufacturers do a good job incorporating risk management into their design, development, and manufacturing processes, but they sometimes fail to enlist other key functions to close the loop. Such departments may include customer service, equipment service, purchasing, distribution, information technology, and, surprisingly, even regulatory affairs. The situation is exacerbated in companies where manufacturing sites are geographically distant from the company’s headquarters. Problems can also arise when companies are organized so that these departments operate at the fringe of an established quality management system (QMS). This often occurs when support departments are more closely aligned with business or marketing operations than with manufacturing operations.

Because ISO 14971 is often perceived as necessary for conducting business internationally, but overkill for complying with U.S. regulations, companies often try to add risk management to their quality management system as a separate, discrete process. The intent of ISO 14971 is just the opposite. If a company operates with a quality management system—and these days a global IVD company must do so—risk management is expected to be fully integrated into all of its processes. Otherwise, serious discontinuities become inevitable. The risk management process of ISO 14971 is an efficient way to satisfy U.S. requirements, as well as those of Europe, Canada, Japan, and many other countries.

Figure 1. (click to enlarge) The process for integrating risk management into the quality control system. Investigations and immediate corrections are typically conducted within the process that caused the deviation.

There are three key components of an effective ongoing risk monitoring process: internal and external surveillance, corrective and preventive action (CAPA), and change control. Fortunately, these are essential elements of a quality management system. There is also a close relationship between risk management and the medical device reporting (MDR) and postmarket vigilance system (see Figure 1).

Postmarket Surveillance

Risk monitoring commences with the start of manufacturing. As part of a product’s launch readiness, manufacturers must have procedures in place to collect, review, and act upon any data that indicate the product may not be meeting its performance specifications. It is a simple matter to extend these procedures to monitor the risk profile of the product, which is the intent of ISO 14971, Clause 9: “Production and Postproduction Information.”

In theory, all the necessary data for risk monitoring should already be available from processes established to monitor the manufacture, distribution, installation, use, and servicing of a device, and from publicly available postmarket surveillance programs.

Complaint trend analysis proves an indispensable tool for identifying design deficiencies and inadequate instructions, as well as unanticipated effects of product or process changes on patient risk. The role of complaint monitoring in risk management is not only to identify new hazards or changes in risk, but also to test the original risk estimates and verify that these are correct. If data show the estimates are wrong, as manufacturers sometimes discover when products enter the real world, the risk analyses must be updated, and previous risk management activities must be reevaluated. Customer feedback can also be used to improve the risk assessment process so that future risk management decisions will be even better. When postproduction information is properly used, risk management becomes a model closed-loop process.

The top row of the diagram in Figure 1 shows several internal and external sources of quality indicators that require monitoring. Each process must have some built-in means of identifying significant product or process failures, investigating their causes, and determining whether a systemic issue exists that should be addressed through CAPA. For risk monitoring, the investigation procedures must include an evaluation to determine whether a failure represents a known hazard, or one that was not foreseen during the original risk analysis.

Risk management may be an imperfect process, but its effectiveness improves with time. Normally, a CAPA is opened when an unanticipated failure mode is encountered and a risk analysis is performed within the CAPA process—ideally, by the same experts who performed the original analysis. The results are evaluated against the predetermined acceptability criteria in the risk management plan. Then, additional risk control measures are applied to reduce the risk (following the hierarchy of risk controls discussed in Part 3 of this series), and the product risk assessment is updated with the results. An investigation should also address the reasons why a hazard was not identified. Does the risk analysis process need improvement? Were the right experts involved? Should the hazard have been identified with the information available?

If the failure mode was already addressed in a risk analysis, then the investigation must ask whether the original risk estimate was correct. Both components of risk—severity and probability of harm—need to be considered. If serious harm or a hazardous situation occurred that was not expected from the hazard, the probability or the severity was underestimated. If the hazardous situation was expected to occur at an acceptably low frequency, then a review of trend analysis data may reveal whether this prediction has been borne out. A CAPA can be opened to address the increased risk and any underlying systemic issues in the risk management process (i.e., what led to underestimation of the risk).

Risk assessments are living documents that must be available when investigating complaints and product failures. Risk monitoring requires a considerable amount of organization and coordination. The Global Harmonization Task Force recommends maintaining a risk table for each product, with entries traceable to the supporting risk assessments, thus making risk information available to those who need it.⁵

A common discontinuity in risk management involves complaint investigations. For example, the front end of the complaint handling system is often delegated to a dedicated group in marketing or customer service skilled in managing customer relations. These professionals are typically responsible for handling all communications with customers. After troubleshooting a customer’s call, those that meet complaint criteria are handed off to the designated complaint-handling unit for follow-up. Unfortunately, the complaints may not be categorized in a way that enables comparison to the failure modes identified in the product risk analyses, making it difficult to correlate this valuable user feedback with the original risk estimates. A modest upfront effort to develop a system that correlates failure categories with the potential failure modes identified during product design significantly reduces the time and effort of evaluating risk when adverse events and malfunctions occur. In addition, such a system can help convince FDA investigators looking into corrections or removals that the quality management system is working and the company is in a state of control.

While each department may collect its own postmarket data, postmarket surveillance is more effective when performed by a cross-functional team, preferably including the same knowledgeable experts that created the original risk estimates—development scientists, design and manufacturing engineers, human factors specialists, user advocates, medical experts, and risk management practitioners. The latter group is required because risk management has become a specialized discipline, and participants must be qualified to perform its functions.

Risk management lends itself to a team approach, but teams are only effective with strong leadership and the support of management. In addition to having the appropriate resources, successful postmarket risk monitoring depends on active participation, clear ownership of follow-up actions, and good performance metrics based on the action outputs and not the data inputs.

Risk monitoring can be conducted for the most part by using existing quality monitoring and investigation processes that feed the company’s CAPA system. In addition, although publicly available information is frequently overlooked or ignored, manufacturers should review these sources as part of the monitoring process—not only regarding their own devices, but also comparable medical devices that could provide insights into possible risks. These reviews will not occur unless specific individuals are assigned to review the data sources and report regularly on their findings. FDA’s Manufacturer and User Facility Device Experience (MAUDE) database is a public source of searchable information about actual and potential adverse events.⁶ Although MDR reports need to be carefully scrutinized to avoid overreaction—manufacturers frequently report events based on limited and inaccurate information—the early warning indicators should not be overlooked. Other external information available to IVD manufacturers includes customers’ quality control data, proficiency survey reports, and clinical laboratory and medical journals that publish product evaluations and user reports.

Relationship of Risk Management to Investigations and CAPA

Companies frequently have trouble defining the relationship of the risk management process to investigations and their CAPA system. This is often because they have not made risk management an integral part of their quality system processes, as illustrated in Figure 1. Investigations of possible malfunctions identified through complaints, service reports, manufacturing defects, engineering nonconformities, supplier audits, and QMS audits must evaluate the risks to patients. For example, a service report may indicate that incorrect results were generated by a laboratory because of a previously unrecognized failure mode. Review of the risk assessment documentation may show that the magnitude of error could be sufficient to affect a physician’s diagnosis and lead to patient harm. In addition to helping decide whether to report the malfunction as an MDR, the risk determination would ensure that root-cause analysis and corrective actions be given higher priority than if the risks were found to be low. The CAPA process coordinates and risk reduction activities to completion tracks.

Controlling Changes

Another important component of postlaunch risk management is a robust change-control system. Product and process changes may be initiated in response to corrective actions, quality improvement programs, or cost-reduction initiatives, as well as interruptions in raw material availability and other reasons outside the manufacturer’s control. Failure to connect change control to risk management is a common discontinuity in quality management systems. IVD manufacturers must be careful not to inadvertently introduce new hazards or increase patient risks when making changes to materials, processes, or product design. In addition, critical risk control measures can be subverted when changes do not take into account the risk mitigation decisions made at the time of product or process design.

Existing change management processes should be easily adapted to include consulting previous risk assessments before a change is made, performing a risk assessment whenever a new hazard or change in risk level is suspected, and updating the risk management file.

Figure 2. (click to enlarge) The integration of risk management into a change-control system.

The diagram in Figure 2 illustrates how risk management is integrated into a change-control system. When changes are proposed, they are evaluated against product and process risk assessments and change history. The change-control process asks whether any new hazards may have been introduced, whether the severity or probability of harm might be increased by the change, and whether existing risk controls will be maintained after the change. If the answer to these three questions is “yes” or “don’t know,” then a risk assessment is required before a decision on the change can be made.

Most manufacturing operations already check their failure mode and effects analyses when contemplating a process change, but purchasing controls and acceptance activities require the same degree of attention. Any decisions to allow changes that could affect the product must include consideration of risk to patients.

An effective change-control system must require that each change be evaluated to confirm it will not increase the probability of a failure that can lead to harm or increase the severity of harm, or affect an established risk control measure. Where applicable, the need to requalify equipment, facilities, or personnel or revalidate a process or test method must be considered because these are important risk controls. Manufacturers must be especially vigilant against risk creep—the cumulative effect that many minor changes may have on the overall risk level.

If the change is being introduced as part of a risk reduction activity, change control must check completion of the verification activities required by ISO 14971 to ensure the effectiveness of risk mitigation. Consulting the risk management documentation related to a proposed change should be an explicit, mandatory part of the change-control procedure.

Purchasing controls are another important aspect of risk management. Supplier qualification criteria and purchase specifications should be based upon risks related to the purchased products and services. Manufacturers should prescribe risk control measures to ensure that purchased product and services meet specified requirements. The degree of purchasing control, and the ratio of acceptance activities to purchasing control, should be commensurate with the residual risk associated with the outsourced product or service.

Conclusion

Although risk management requirements have been in effect in the United States for 10 years, and the European IVD Directive became fully effective in 2003, compliance audits and discussions with industry colleagues indicate that many IVD manufacturers are still struggling to integrate risk management into their quality management systems.^7,8 This situation is not unique to the IVD industry, but the indirect risk to patients adds a challenging dimension for IVD manufacturers.

The final installment of this series will discuss the relationship of risk management to the MDR and postmarket vigilance systems, and it will highlight gaps sometimes found in ISO 14971 risk management programs. It will also discuss changes in the second edition of ISO 14971, which is scheduled for publication in 2007.

---