Medical Device Link.

Originally Published IVD Technology September 2003

REGULATIONS & STANDARDS

Richard Miller is staff scientist in Consumables Manufacturing at Dade Behring Inc. (Deerfield, IL). He can be reached via e-mail at Rick_R._Miller@dadebehring.com.

One of the essential requirements of the IVD Directive can be found in the second paragraph of section A3 in Annex I: “The traceability of values assigned to calibrators and/or control materials must be assured through available reference measurement procedures and/or reference materials of a higher order.”¹ This statement is the technical traceability requirement of the directive. Besides this statement, the directive itself provides few other references to traceability and little guidance on how to meet this requirement. The purpose of this article is to describe how IVD manufacturers should go about meeting this traceability requirement. 

What Is Metrological Traceability?
 
The objective of traceability in analytical measurement is to ensure that laboratory results can be traced back to standards and methods that are recognized as accurate. In other words, if all of the analytical results are traceable to well-defined materials and methods, the results obtained will be the same, within a known margin of error. 

Traceability is envisioned as one way of obtaining consistent test results from laboratory to laboratory. It is often recognized that the interpretation of these results can be based on decision values obtained from large studies. These decision values may be published and recommended to physicians with no reference to the specific methods used to determine them.

However, today’s laboratories may or may not provide test results using methods whose results are equivalent to those obtained in prior studies. For test result information to be useful in these cases, all of the methods used by laboratories must be comparable. Some of the differences in results can be substantial and could potentially change the decisions made by physicians. Additionally, in today’s mobile society, physicians may receive results from several different laboratories with little or no knowledge about how the different methods that are used can be compared. 

Several studies have demonstrated how laboratory results can be less accurate than might be desired. One such study by the College of American Pathologists (CAP; Northfield, IL) examined the accuracy-based-evaluation criteria. The evaluation criteria section of a chemistry survey report contains a section on “Accuracy-Based Evaluation” and “Medical Usefulness Limits.” With this section, CAP set proper goals for accurate results and compared them with the current evaluation criteria. For the nine analytes examined in this study, the results from the evaluation criteria were greater than the CAP goals in all but one case. The Institute for Reference Materials and Measurements (IRMM; Geel, Belgium) also completed a similar study of 1100 laboratories worldwide. In this study, IRMM demonstrated significant deviations from “desirable total error” for calcium and sodium.² 

Providing test results that are traceable is one way to improve this situation. Some examples of successful models of traceable reference systems that have improved the harmonization of results are those for glucose, using the CDC reference method, and cholesterol, using the Abell-Kendall method. Both of these reference systems also include the use of appropriate reference materials from the National Institute of Standards and Technology (NIST; Gaithersburg, MD). 

The concept of traceability and its ability to harmonize test results have been discussed and written about for decades. One example is the publication of the “National Understanding for the Development of Reference Materials and Method for Clinical Chemistry” in 1978 by the American Association for Clinical Chemistry (Washington, DC).³ Although the science of metrology has evolved during the last three decades, the basic principle of harmonizing results using recognized reference materials and methods is still understood as a necessary one.
Traceability Standards

Figure 1. The responsibility of the manufacturer for metrological traceability shall begin at the assigned value for a product calibrator and end at the secondary calibrator or secondary reference measurement procedure as the case may be and if such exist. (The former segment is delimited by two horizontal broken lines). The manufacturer, however, shall also be responsible for the intructions for use. Metrological traceability of a value assigned to a trueness control material shall utilize the calibration hierarchy shown as appropriate, substituting in i) with manufacturer’s product trueness control material.

The IVD Directive states that IVD manufacturers must assure their tests are traceable to reference methods and materials of a higher order. However, the directive does not explain how manufacturers should attain this objective. 

In order to provide guidance on this subject matter, the European Commission asked the European Committee for Standardization (CEN) through one of its technical committees to develop guidance standards on traceability. This committee (CEN TC140) has been working closely with the International Organization for Standardization (ISO) technical committee for clinical laboratory testing and in vitro diagnostic test systems (ISO TC212) to develop these standards for meeting the traceability requirements. 

As a result, both organizations have developed and approved five traceability standards.^4–8 These standards are voluntary in that IVD manufacturers have the option to use other technically acceptable processes to establish traceability. However, a number of technical experts on traceability developed these standards that provide accepted technical guidance on how to meet the requirements under the IVD Directive. If manufacturers decide to develop and use another technically acceptable means of establishing traceability, they should be prepared to defend their process and procedures on technical grounds. 

Of these five standards, two of them address the overall principles of traceability: EN/ISO 17511 and EN/ISO 18153. The primary difference between these two standards is that EN/ISO 18153 describes acceptable systems for the measurement of enzyme activity and EN/ISO 17511 describes acceptable traceability systems for all other quantitative clinical measurements. Although these standards have some differences, they are similar enough for the purposes of this discussion to be considered the same. The other three standards are specific specialized documents describing reference methods and materials of a higher order, and the laboratories that would perform these reference methods. 

Developing a Traceability Chain

Traceability can have many definitions. An IVD manufacturer may consider its ability to track a raw material through its manufacturing system as traceability. However, metrological traceability, as it applies to EN/ISO 17511, is “the property of the result of a measurement or the value of a standard whereby it can be related to stated references, usually national or international, through an unbroken chain of comparisons all having stated uncertainties.” While this definition is not restricted to medical measurements, it is accepted throughout the measurement laboratory industry regardless of the types of measurements. 

So what does this definition mean? Most measurements are expressed by certain physical constants such as weight, time, temperature, etc. Traceability is the mechanism to track the path of a measurement from these constants, as defined by the International System of Units (SI), to the actual result. 

Taking sodium in serum as an example, this principle can be illustrated by developing a traceability chain. In this case, the analyte or measurand is sodium in serum in molEq/L. Therefore, the defined constants, or SI units, are volume (L), weight (g), and the gram equivalent weight of sodium. 
A chain of traceable comparisons is developed by starting with the SI units and transferring them to the final result. In the case of sodium, highly purified sodium chloride is often utilized in these comparisons. In this example, the material used is NIST standard reference material (SRM) 919a. To determine the purity of the sodium chloride used in SRM 919a, NIST utilizes primary reference methods: a coulometric method for chlorine and a gravimetric method for sodium. This represents the first set of comparisons in the chain. 

An IVD manufacturer may then find it desirable to use a serum-based calibrator to calibrate the instrument for analyzing sodium in serum. If the system is designed in this manner, the manufacturer will analyze this calibrator with a particular method. 

In this hypothetical case, the manufacturer, using SRM 919a, properly calibrated or certified volumetric flasks, and calibrated balances, prepares standard solutions. These solutions standardize a flame photometer with a reference-method procedure. 

The flame photometer then assigns the calibrators used to calibrate the instrument, and a second set of comparisons is established. From the SI units to SRM 919a, to the serum-based calibrator, the values assigned to the calibrator are traceable through these links. 

Using this instrument and calibrator, a laboratory can follow the traceability chain for a sample being analyzed. That sample is traceable to the SI unit through an unbroken chain of comparisons. 

Regarding the last part of the traceability definition (“all having stated uncertainties”), uncertainty is the amount of dispersion or random error that can be expected using the manufacturing or analytical processes at each link of the chain. Each link in this chain will have a known uncertainty that can be expressed in appropriate units. 

Documenting and Demonstrating Traceability 

The IVD Directive requires manufacturers to document various factors during the development and manufacture of their products, and account for how they reached the intended objectives. Traceability is no different. Both EN/ISO 17511 and EN/ISO 18153 provide a significant amount of detail about how manufacturers should document and validate traceability in their products. 

In documenting traceability, one of the first requirements is that IVD manufacturers must maintain the same measurand throughout the analytical process. A measurand is a thorough description of what is being measured and is different from an analyte. 

For example, one measurand could be the activity of CKMB in plasma or serum. Another measurand could be the mass of CKMB in plasma or serum. While the first one is measuring catalytic activity, the second is measuring the amount of substance independent of activity. Since these two measurands are different, adequate traceability information cannot be provided by interchanging materials and methods that cross these lines. Manufacturers must therefore assure that the measurand remains constant throughout the traceability chain. 

A traceability chain model shows the possible steps in the process that need to be documented (see Figure 1). Several things can be observed from this figure. First, traceability demonstrates the need to track the chain from the most fundamental unit all the way to the individual patient result. 
Second, two horizontal dashed lines are present in the figure. Above the upper dashed line, national and international metrology institutes, such as the International Bureau of Weights and Measures (BIPM; Paris), NIST, and IRMM, are responsible for establishing standards for that part of the traceability chain. Below the lower line, the end-user or the laboratory reporting the results is responsible for documenting traceability. IVD manufacturers are responsible for those parts of the chain with the designation “ML”. 

Third, the graphic at the far right of the figure demonstrates the effect of uncertainty at each level of the traceability chain. As the chain progresses, uncertainty increases because each step includes the uncertainty from the previous step. The more steps in the chain, the greater the uncertainty. To understand the total uncertainty at the final patient-result level, the amount of uncertainty contributed at each step needs to be determined. Each national metrology institute must provide its contribution to uncertainty in its measurements and reference materials. 

IVD manufacturers must take into account those measurements and reference materials when they determine the amount of uncertainty in their final calibrators. And end-users must take the manufacturer’s uncertainty into account when determining the uncertainty of the final results. This traceability chain model enables clinical laboratories and physicians to understand how results from two different methods or laboratories are comparable. 

Taking the traceability chain into account, one of the primary requirements for IVD manufacturers is to provide information about how the values assigned to their calibrators can be traced back to the most fundamental unit and the related uncertainty. Manufacturers of calibrators or certain types of controls must provide this information to customers upon request. 

To meet this requirement, ISO standards call for the validation of metrologically traceable calibration. In section 7 of ISO 17511, three major elements of this validation process are described. These elements are intended to assure that the results obtained by the end-users are consistent with the reference methods and materials used throughout the entire traceability chain. 

As discussed above, the first element is the measurand that is measured by the reference procedures. When used by the routine procedure, it should be the same. 

The second element is IVD manufacturers’ need to validate that the results obtained by the routine procedures and calibrated with their calibrators are mathematically the same as the results from the reference measurement procedures for relevant human samples. Although the standard goes into more detail about this requirement, the goal is for the manufacturers to have on file results that compare the reference method procedures with the routine procedures and demonstrate they both end up with essentially the same results. This requirement is described as the commutability of the product calibrator. 

The third element is a description of the validation requirements for what the standard calls the manufacturer’s working calibrator. The manufacturer’s working calibrator is the calibrator that a manufacturer uses to assign product calibrator values. This working calibrator has a different commutability requirement, which is if a working calibrator is included in the validation process mentioned above, it must have the same relationship as a patient sample. Although this relationship is not a requirement for the product calibrator, it is specifically mandated for working calibrators. 

The remaining portions of section 7 of ISO 17511 describe the use of statistical methods to obtain results, but do not necessarily mandate specific statistical methods or criteria. These decisions are left to the manufacturers to define in their validation processes. 

Reference Methods and Materials

The ability to document traceability is possible only if certain resources are available to IVD manufacturers. Those resources include internationally recognized reference materials and methods, and the ability to use them in every measurand for every manufacturer’s calibrator or trueness control. Of course, this assumes that manufacturers can find the reference methods and materials that exist and that they are appropriate for use in the intended traceability chain.

The other three traceability standards authored by ISO TC212 and CEN TC140 are the following: 

• ISO 15193/EN 12286, in vitro diagnostic systems. Measurement of quantities in samples of biological origin and presentation of reference measurement procedures.
• ISO 15194/EN 12287, in vitro diagnostic medical devices. Measurement of quantities in samples of biological origin and description of reference materials.
• ISO 15195, laboratory medicine. Requirements for reference measurement laboratories. 
These standards specifically describe what is required of internationally recognized reference materials, methods, and laboratories. By comparing each material, method, and laboratory with these requirements, IVD manufacturers can decide which of them meets these requirements. This can be an extensive process and may lead to significantly different decisions by different manufacturers. 

Recognizing the difficulties IVD manufacturers may encounter in obtaining and qualifying these resources, the International Federation of Clinical Chemistry and Laboratory Medicine (Milan, Italy), BIPM, the World Health Organization (Geneva), and the International Laboratory Accreditation Cooperation have formed the Joint Committee on Traceability in Laboratory Medicine (JCTLM) to assist with this process.^9–13 This committee started its work in June 2002, and intends to provide lists of reference materials and methods that meet ISO standards in the fall of 2003. Work on reviewing and certifying reference laboratories is also ongoing.

Since IVD manufacturers require a certain amount of time to implement any standards or methods they are unfamiliar with, they are forced with deciding whether they should certify these materials themselves or wait for international recognition. Even after JCTLM provides its lists, this dilemma must reside with the individual manufacturer since the applicability of those standards and methods must be determined on a case-by-case basis. However, the lists will provide the international recognition required as well as a ready resource for identifying such materials, methods, and laboratories. 

Another resource needed by IVD manufacturers is a mechanism for determining uncertainty. The traceability standards refer to a guide to the expression of uncertainty in measurement.¹⁴ This document provides a description of the principles for determining uncertainty. Since it describes these principles in general details that are applicable to many measurement systems, other resources are available that are more applicable to chemical measurement and may be more understandable.^15–17 These resources describe the calculation of uncertainty in detail in a language more directly applicable to the types of measurements performed by medical laboratories.

There are also many cases in which reference materials and measurement procedures do not exist. For example, of the 400–600 measurands that the standards state exist, only about 30 to 50 have recognized reference methods. Although there are many more reference materials, there are many more measurands and analytes which have no appropriate reference materials. When discounting such materials that do not apply to the same measurand, the list gets even smaller. One example is the application of biological-activity standards to chemical or immunochemical measurements.

The premise of the standards and the IVD Directive is that if there are no applicable methods or materials, the measurements cannot be made traceable to them. IVD manufacturers are then able to use their technical judgment to prepare and utilize what they think is appropriate. This does not imply that standards cannot be applied, but the directive does not require it. If a manufacturer chooses to use the standards, the traceability chain is simply shortened. It will start with the standard the manufacturer has defined as appropriate. This may even start with a defined unit that has an uncertainty of zero. 

Conclusion

The IVD Directive and the ISO/CEN standards provide a uniform system for describing the traceability of analytical values at the laboratory level. However, they focus on values assigned to calibrators provided by IVD manufacturers. Such a system is expected to provide more-harmonized results for clinical measurements and improve medical decisions. Unfortunately, although the resources are unavailable to implement completely such a system for all of the types of analytes measured, it is hoped the system will improve the measurement of those analytes for which such resources do exist, and provide a forum for developing additional resources where they do not yet exist. 

References

1. “Directive 98/79/EC of the European Parliament and of the Council of 27 October 1998 on In Vitro Diagnostic Medical Devices,” Official Journal of the European Communities L 331 (1998): 1–37.

2. International Measurement Evaluation Program IMEP-17 Trace and Minor Constituents in Human Serum (EUR 20694 EN) “Report to Participants, Part 2” [cited 12 August 2003]; available from Internet: www.imep.ws/maintenance.htm?aspxerrorpath=/. 

3. J Boutwell, ed. National Understanding for the Development of Reference Materials and Method for Clinical Chemistry, Conference Proceedings (Washington, DC: American Association for Clinical Chemistry, 1978).

4. In Vitro Diagnostic Medical Devices, Measurement of Quantities in Biological Samples, Metrological Traceability of Values Assigned to Calibrators and Control Materials, ISO 17511 [cited 12 August 2003]; available from Internet: www.nccls.org. 

5. In Vitro Diagnostic Medical Devices, Measurement of Quantities in Samples of Biological Origin, Metrological Traceability of Values for Catalytic Concentration of Enzymes Assigned to Calibrators and Control Materials, ISO 18153 [cited 12 August 2003]; available from Internet: www.nccls.org. 

6. In Vitro Diagnostic Systems, Measurement of Quantities in Samples of Biological Origin, Presentation of Reference Measurement Procedures, ISO 15193, 18153 [cited 12 August 2003]; available from Internet: www.nccls.org. 

7. In Vitro Diagnostic Medical Devices, Measurement of Quantities in Samples of Biological Origin, Description of Reference Materials, ISO 15194 [cited 12 August 2003]; available from Internet: www.nccls.org. 

8. Laboratory Medicine, Requirements for Reference Measurement Laboratories, ISO 15195 [cited 12 August 2003]; available from Internet: www.nccls.org. 

9. The International Federation of Clinical Chemistry and Laboratory Medicine Web site [cited 12 August 2003]; available from Internet: www.ifcc.org/ifcc.asp. 

10. Bureau International des Poids et Mesures Web site [cited 12 August 2003]; available from Internet: www.bipm.fr/enus/.  

11. World Health Organization Web site [cited 12 August 2003]; available from Internet: www.who.int/en/. 

12. International Laboratory Accreditation Cooperation Web site [cited 12 August 2003]; available from Internet: www.ilac.org. 

13. Joint Committee on Traceability in Laboratory Medicine Web site [cited 12 August 2003]; available from Internet: www.bipm.fr/enus/2_Committees/JCTLM.shtml  

14. Guide to the Expression of Uncertainty in Measurement, 1st ed., (Geneva: International Standards Organization, 1993).

15. EUROCHEM/CITAC Guide Quantifying Uncertainty in Analytical Measurement [cited 12 August 2003]; available from Internet: www.measurementuncertainty.org. 

16. Guidelines for Evaluating and Expressing the Uncertainty of NIST Measurement Results Technical note 1297 (Gaithersburg, MD: National Institute for Standards and Technology, 1994).

17. SK Kimothi, The Uncertainty of Measurements, (Milwaukee: Quality Press, 2001).  

Copyright ©2003 IVD Technology