Medical Device Link.

Originally Published IVD Technology November/December 2005

REGULATIONS & STANDARDS

Donald M. Powers and Timothy Roscoe

Donald M. Powers, PhD, (top) is president and principal consultant of Powers Consulting Group (Pittsford, NY), and Timothy Roscoe (bottom) is director of communications at the Clinical and Laboratory Standards Institute (Wayne, PA). The authors can be reached at powers@frontiernet.net and troscoe@clsi.org.

In the spirit of collaboration and consensus, the Centers for Medicare and Medicaid Services (CMS) has been seeking broad stakeholder input on the degree of quality control (QC) that is necessary for laboratory testing regulated under the Clinical Laboratory Improvement Amendments (CLIA). In support of this initiative, the Clinical and Laboratory Standards Institute (CLSI, formerly NCCLS; Wayne, PA) sponsored a workshop to define the issues and explore appropriate QC strategies for commercial IVD systems.

The CMS initiative emerged after the final CLIA regulation was released in January 2003.¹ The final regulation introduced equivalent quality control (EQC), which was discussed in an article published in IVD Technology (March 2005).^2,3 The term EQC refers to an alternative QC scheme that takes into account a lab system's ability to monitor the performance of all or part of the testing process. The EQC concept established that the more the process is monitored by the system, the less external QC testing is required. Three EQC options are allowed by CMS (see Table I).

EQC has generated substantial controversy in the clinical laboratory community. It was introduced to address concerns about the cost of traditional QC practices, which seemed excessive for modern lab systems designed to be self-monitoring. However, critics pointed out that the reduced QC testing levels prescribed in the CLIA final rule were not based on a scientific rationale. They also questioned whether EQC would provide the laboratories with adequate confidence that incorrect results would be detected.

To obtain input on these issues and possible solutions, CLSI organized the QC for the Future workshop to maximize the involvement of key stakeholders. In addition to presentations by invited government, industry, and clinical laboratory representatives, workshop participants convened in breakout sessions to prepare consensus recommendations for CMS to consider. This article presents workshop highlights that are of particular interest to IVD manufacturers.

CLIA QC History

The controversy over attempts to stipulate QC rules suitable for all commercial IVD systems dates back to CLIA's inception. The initial proposed rules were based on QC schemes developed primarily for batch testing using manual kit assays and automated continuous- flow systems. At the time the QC rules were introduced, such IVDs were already being replaced by more-stable unit use and random-access technologies.⁴ The proposal's perceived negative impact on laboratory costs and technological innovation led to NCCLS' CLIA Congress of 1990 (NCCLS became CLSI in 2005). This congress produced three days of testimony and evidence regarding the amount of QC needed to protect the safety of patients. In the end, a consensus enabled the Health Care Financing Administration (HCFA) to modify the linearity testing requirements and reduce the number and frequency of required QC samples (HCFA became CMS in 2001).

The stark contrast between that earlier era and today's more collaborative regulatory climate is worth noting. On the advice of their legal counsel, FDA prohibited representatives from the Centers for Disease Control and Prevention (CDC) and HCFA, the agencies involved in the CLIA rule making, from attending the 1990 CLIA Congress. This time, CDC, CMS, and FDA have invited stakeholders to provide constructive, data-driven, evidence-based, and scientifically sound feedback that the agencies can use to revamp the CLIA quality requirements.

During the workshop, Joe Boone, PhD, associate director for science in the division of public health partnerships at CDC, gave a historical overview of laboratory QC practices. He discussed the dilemma that regulators face in developing QC requirements suitable for the wide spectrum of lab systems, testing environments, and personnel skills. He stressed the importance of considering operator, environment, and patient population issues, in addition to internal QC checks built into lab systems, when designing a holistic QC process.

Boone also emphasized the role of IVD manufacturers in providing adequate information about the components monitored by built-in QC. He said users find QC instructions for some lab systems difficult to follow because they are located throughout the product literature and contain ambiguous or insufficient information. Insisting that determining appropriate QC must be based on scientific evidence, he acknowledged that manufacturers can help by providing the initial data. However, in the end, laboratories must be held responsible for the long-term data collection so that operator and environmental variation are reflected.

QC for Point-of-Care Testing

Valerie L. Ng, MD, PhD, professor and interim chair of the department of laboratory medicine at the University of California at San Francisco and director of the UCSF clinical laboratory at San Francisco General Hospital, gave a presentation on the special QC requirements for point-of-care testing (POCT). Such special requirements are necessary because POCT may be performed outside the laboratory or in physicians' offices without oversight. Recent CMS surveys confirmed earlier POCT studies that documented inadequate QC and quality issues such as failure to follow IVD manufacturers' instructions. In addition, high personnel turnover often results in serious training and proficiency issues. Consequently, manufacturers need to design POCT devices that are immune to operator and environmental influences.

Ng acknowledged that the analytical portion of the QC process has improved dramatically during the past 20 years, largely due to technological innovations, and that most QC errors were associated with the pre- and postanalytical phases. She also pointed out that while electronic QC might be adequate for testing the inner workings of a lab instrument, it is not sufficient as a sole QC measure. She offered IVD manufacturers the following recommendations to enhance the robustness of QC in their POCT devices.

• Include internal QC for all unitized devices and print simple step-by-step test instructions on the cartridge.
• For tests that require results to be read in a specified time frame, design the tests to become unreadable once the time period has expired.
• Include security features that lock out test personnel if their identification is incorrect or patient information is not entered.
• Include the necessary connectivity for instruments.
• For devices that require specific storage conditions, design the test to become unusable if a recommended storage condition is exceeded.
• Make liquid QC material available for POCT personnel to verify instrument linearity and reportable range, and to perform external checks.
• Compare POCT patient values with those obtained by a reference method.
• Provide timely, helpful, and technical expert customer service.

Ng also cautioned that practical considerations could limit the ability of laboratories to adopt extended QC intervals. Although some analytical lab systems might require QC only once a month, other factors could dictate more-frequent checks for pre- and post-analytical errors. Such factors include specimen instability (e.g., the degradation of analytes upon storage), the length of time that specimens will be retained in laboratories for possible reanalysis, and the risk that physicians will act on out-of-control test results before they can be repeated.

James Westgard, PhD, president of Westgard QC Inc. (Madison, WI), added to the discussion by challenging assertions that the performance of laboratory tests is good enough for medical needs. He cited examples of several important analytes that failed to meet the six-sigma standard of excellence in proficiency surveys. He urged caution in relaxing current QC standards until the medical requirements for accuracy and precision of lab test results are better defined. During the breakout sessions, there was general agreement that more efforts should be applied to understanding the performance requirements of lab tests for medical uses.

Technology Trends

Innovations in IVD technologies have continued to advance at a rapid pace during the post-CLIA era. Such innovations were driven by user requirements for simpler, more-reliable lab systems that were economical to operate, and the FDA quality system regulation (QSR), which mandated IVD manufacturers to implement design controls and risk management processes.⁵ Both factors led to a greater emphasis on preventing incorrect test results by systems and detecting incorrect results before they are reported by laboratories.

Fred Lasky, PhD, director of regulatory affairs at Genzyme Diagnostics (Cambridge, MA), summarized IVD manufacturers' efforts to incorporate internal checks that lessened the need for external QC testing. He stated two possible approaches for achieving quality test results: monitoring to detect problems and then correcting such problems in follow-up efforts (quality control); or establishing confidence that the user meets performance requirements (quality assurance).

At the heart of the EQC concept are the internal QC mechanisms that IVD manufacturers have incorporated into their lab systems. Since analytical systems are extremely diverse, it would be impossible to describe all the novel QC monitoring technologies that manufacturers have developed. In his presentation, Lasky provided the following examples of built-in QC approaches that monitor internal lab instrument processes.

• Bar coding to identify and manage patient samples, reagent lots, and calibration information.
• Appropriateness checks on optical or impedance readings in spectrophotometric and potentiometric measurements.
• Sensors that verify reagent volumes and reservoir levels.
• Leak detectors and valve monitors in reagent delivery tubing.
• Automatic detection of reagent deterioration.
• Positive reagent delivery checks to ensure that pumps send the right liquids to the right place at the right time.
• Response checks to verify the correct signal-to-noise ratio.

In discussing future challenges, Lasky cited QC for emerging IVD technologies such as microarrays, indwelling monitors, and molecular assays. While conceding that no lab system would be able to guarantee test results, he emphasized that the surest path to achieve quality is through active partnerships among healthcare providers, clinical laboratories, IVD manufacturers, and patients.

QC and Risk Management Concepts

Regulations in the United States and the European Union apply risk management principles to IVD medical devices.^5,6 According to these regulations, IVD manufacturers are required to analyze and evaluate the risks associated with their products, and take appropriate measures to reduce and control the risks to acceptable levels. Since incorrect test results for many analytes can present serious risks to patients, manufacturers incorporate features in the designs of their lab systems for eliminating the potential causes of incorrect results (e.g., nonspecificity, bias, imprecision, nonlinearity). Manufacturers also add detection mechanisms to alert users when results may be compromised by a system problem. When detection mechanisms monitor some portion of the testing process, they are considered built-in QC checks, which the EQC concept has been trying to acknowledge.

Most global IVD and medical device manufacturers follow the risk management process outlined in ISO 14971.⁷ This standard considers QC to be a risk control measure since it is intended to reduce the likelihood that an incorrect result might be reported.

Donald Powers, PhD, an industry consultant and convener of ISO TC212's working group 3, proposed extending the IVD manufacturer's risk management process to the clinical laboratories through a partnership. He suggested specifically using risk assessment as the basis for deciding what further risk controls are needed to avoid reporting incorrect test results (i.e., mitigating the risk of incorrect results to patients). Laboratories would use such structured risk assessment tools to identify the most likely places in their testing process that a failure could occur and lead to incorrect results.

IVD manufacturers can assist laboratories by providing information about the risk controls built into their systems. Guided by the risk assessment, laboratories would implement preventive measures to avoid failures from occurring, if technically and financially possible, or implement suitable detection methods (e.g., quality control testing at appropriate concentrations and frequencies). Laboratory directors would make risk management decisions based on the severity of the harm that could occur from incorrect test results and the probability that such harm would occur, as outlined in the ISO risk management standard.⁷

EQC Option 4

A fourth EQC option, which IVD manufacturers proposed through AdvaMed (Washington, DC), would allow manufacturers to establish the scientific validity of a QC procedure designed for their lab systems. Luann Ochs, director of regulatory submissions at Roche Diagnostics (Indianapolis) and chair of AdvaMed's CLIA working group, described the proposal in her presentation. The proposal consists of the following three steps.

• A manufacturer performs a scientifically rigorous risk assessment and validates a QC procedure that will provide substantial equivalence to traditional QC.
• The manufacturer submits the validation data to FDA using existing submission processes. FDA reviews and approves the proposed QC procedure if the validation data are considered sufficient.
• Laboratories will be allowed to use the alternative QC protocol in place of the CLIA-mandated QC requirements for that assay.

The EQC option 4 proposal is the subject of a CLSI document being developed as an outgrowth of the QC for the Future workshop. This document will provide guidelines for validating the effectiveness of risk control measures identified during a risk assessment. IVD manufacturers can then submit the risk assessment and validation data to FDA for review. The built-in QC cleared by FDA can be used by laboratories to satisfy all or part of the QC requirements for a given test. As with the other alternative QC options introduced in the CLIA regulations, laboratory directors ultimately bear the responsibility for decisions regarding alternative QC in their individual laboratories. If option 4 is accepted by CMS, CDC, and FDA, options 1–3 are expected to be eliminated.

During the EQC option 4 breakout session, participants urged the rapid completion of the CLSI document and other new QC alternatives, and emphasized the importance of keeping them flexible to leave room for emerging IVD technologies. The participants suggested that lab devices should be tested by the intended users in their intended working environments to ensure proper training of operators in using the devices. Attendees also stressed the importance of managing risk, using a process map, and modeling failures.

Conclusion

The consensus process for developing EQC option 4 is already under way through a CLSI subcommittee. Breakout session participants suggested that options 1–3 should be discontinued upon successfully implementing option 4, a decision that CMS will make once the new guidelines have been completed. The CLSI subcommittee on the validation of risk mitigation is scheduled to meet in Washington, DC, this fall, to discuss issues related to EQC option 4 and develop a detailed plan for the related CLSI document. The 60-member subcommittee holds the potential for developing a valuable consensus document to aid in the regulatory process.

Meanwhile, interest in the potential application of risk management principles to clinical laboratory testing is picking up. TC212's working group 1 is developing a technical report with guidelines for integrating risk management into a laboratory's ISO 15189 quality management system. CLSI is devoting the main workshop at its 2006 meeting to the use of risk management tools to reduce laboratory errors and improve patient safety. All of these activities are geared to a more holistic approach to quality control and quality assurance in the laboratory.

References

1. Code of Federal Regulations, 42 CFR 493.
2. "Clinical Laboratory Improvement Amendments (CLIA)—Equivalent Quality Control Procedures, Brochure #4," the Centers for Medicare and Medicaid Services Web site (Baltimore: 2003 [cited 27 September 2005]); available from Internet: www.cms.hhs.gov/clia/6606bk.pdf.
3. MB Bandy, "CLIA Quality Control Requirements: An Opportunity for IVD Companies," IVD Technology 11, no. 2 (2005): 24–29.
4. Federal Register 57 FR:7002–7186, February 28, 1992.
5. Code of Federal Regulations, 21 CFR 820.
6. "Directive 98/79/EC of the European Parliament and of the Council of 27 October 1998 on In Vitro Diagnostic Medical Devices," Official Journal of the European Communities L 331 (1998).
7. "Medical Devices—Application of Risk Management to Medical Devices," ISO 14971:2000 (Geneva: International Organization for Standardization).

Copyright ©2005 IVD Technology