IVD Technology
Magazine
IVDT Article Index
Originally published September, 1997
Formal experimental design and analysis for immunochemical product development
Part 1
John A. Wass
A structured approach to experimentation can help IVD manufacturers improve products while streamlining the development process.
In contrast to some researchers who are able to take a more leisurely approach to their studies, the industrial scientist is under constant pressure to produce products that meet exacting specifications with the fewest resources. To adhere to rigorous product requirements, the industrial statistician must employ those design and analytical tools that allow the most information to be squeezed from the least effort. This need applies particularly to development of technical products such as medical diagnostics, where reagent lots must be constantly available and produced to rigid specifications.
This brief introduction to the design of experiments (DOE) focuses on design tools that minimize resource requirements and optimize product specifications. Part 2, which will appear in a subsequent issue, presents an immunodiagnostic application of these tools as well as a brief analysis to illustrate the methodology.
DOE and Design Types
Formal experimental design has several advantages over the classic, one-step-at-a-time approach that has been favored by academe. These advantages include improved performance characteristics, reduced costs, and shortened development and testing times.
The formal experimental design examines changes in output variables for a process, given any combination of input variables. In the most basic immunochemical designed experiment, various concentrations of biochemical reactants (for example, an antigen and an antibody) are mixed and the resulting reaction rate or concentration of product is measured. The researcher can then ensure a desired rate or concentration to within acceptable specifications by setting the antigen and antibody concentrations to those values derived by the analysis.
By isolating and better understanding those factors that most affect the outcomes, the researcher can devote fewer resources to investigating the less-important factors. In testing design strategies or troubleshooting established ongoing processes, the experimental design process may be used to gain additional knowledge about the relationships among input and output variables. Formal design also allows the researcher to mathematically model the process and optimize the response.
The goal is to fill a design space with the least number of points (data collection units), yet still fully characterize the system. Researchers accomplish this goal by employing a number of well-defined design types that are standard in the literature. For more adventurous experiments or for modeling a space sufficiently different from the norm, special models are available that are easily implemented in several commercial software programs.
In all of these experiments, the strategy is essentially the same. The first experiment is a screening run. This run isolates those input factors that are most important to the outputs. Tools available for this experiment include statistical output from the software and the researchers' own knowledge of the system, including the cost factors.
The second step is exclusion of those factors deemed to be of minor significance to the desired output, based on the results of the screening experiment. With the remaining factors, researchers can use a response surface design to determine the changes in outputs for any combination of changes in inputs. Software analysis allows the response to be targeted to a given value within the limits of experimental error. The outputs may also be simultaneously maximized or minimized. For instance, cost may be set as a factor to be minimized. In addition, many software packages allow visualization of this three-dimensional response surface, rotation of it, and examination of specific points.
The last experiment is a verification run, whereby the outputs are checked against the predictions of the response surface design. Once the suitable input settings have been verified, the process may run at these settings until any inputs are changed.
To visualize the conversion of measurable, real-world inputs and outputs to a response surface, consider the following example. Typical immunochemical input factors include antigens, antibodies, and fluorescently labeled probes. In Figure 1, a simple relationship between the antigen and antibody concentrations (the inputs) and the reaction rate being measured (the output) is illustrated. Since we can envision only three dimensions, the concentration of the probe molecule has been held constant. Readily measured concentrations of the inputs are plotted on the x- and z-axes, while the resultant reaction rate is given on the y-axis.
Figure 1. A response surface. Axis labels represent arbitrary units.
The response surface consists of the three-dimensional collection of all experimental points measured and plotted (as well as the areas defined by these points) and is bounded by the limits of the actual experiment. These bounds are called the design space.
Screening Designs
The screening run may be designed using one of the following models:
Linear. A linear screening design consists of a line connecting end points that represent only those values near the highest and lowest settings of each input. The limitation of this design lies in what may happen in the center of the design space, perhaps a bend or curvature in the area where no data have been taken. This design is illustrated in Figure 2, in which the reaction rate is plotted against the change in antigen concentration while the antibody concentration is held constant. In statistical terms, the antibody is held at a single level.
Figure 2. Linear screening design.
Linear with Center Point. Although the linear-with-center-point design is linear, it does require data collection at the center (see Figure 3). This requirement eliminates the drawback of the strictly linear design. These are among the most popular and widely used screening designs.
Figure 3. Linear-with-center-point screening design.
Plackett-Burman. Two-level screening strategies, Plackett-Burman designs are mathematically derived in multiples of four trials (that is, 8, 12, 16, 20 trials and up). The term level here may be thought of as the different concentrations of the reagents used. Therefore, researchers may enter the concentrations within a software package as either high or low, or use the actual numerical values (for instance, 10 µM/dl and 20 µM/dl). If one trial is allowed for a constant, the remaining number of trials is 7, 11, 15, 19, or more. To use a design for variables not occurring in this sequence, the researchers may choose a larger design and ignore the unused variable points in the design space.
These designs have two main drawbacks. Lack of fit is difficult to assess, and first-order effects may be confounded with interaction effects (e.g., it may be impossible to separate an antigen effect from an antigen-antibody interaction effect).
Response Surface Designs
A variety of response surface designs are available for visualization of the effects of changes in inputs.
Factorial Designs. These occur as two types, full and fractional. Full factorial designs are used to estimate all possible effects, including interactions, of the input variables. The number of runs for a design that uses k variables and n levels is nk. This equation means that the number of runs increases dramatically as the numbers of levels and factors increase. The advantages of the full factorial design include orthogonality (ability to exclude confounding, among other properties), lack of aliasing (identical columns in the design), and evaluation of all main effects and interactions. The disadvantages include time, cost, and resource commitment.
Fractional factorial designs retain orthogonality while requiring fewer runs. However, doing fewer runs means acquiring less information. To ensure that subsequent runs capture the most important information, experimenters usually eliminate runs for interactions that are deemed insignificant.
Quadratic Designs. These are based on a standard quadratic model:
They therefore contain linear, interactive, and quadratic terms. These designs are easily implemented with software, and the mathematical results are most readily associated with physical events or response surface curvature.
Partial Cubic Designs. The partial cubic model includes all the terms in a quadratic model but also includes terms having cubic interactions:
It does not contain pure cubic terms. Since cubic interactions occur only rarely in chemical products development work, this exclusion is usually justified.
Box-Behnken Designs. An efficient three-level design that uses embedded 2k designs, the Box-Behnken design holds certain factors at their center points. Tables that specifiy the number of runs to be used are published and readily available. The model drops all corner points in the design space. This elimination of data may be of concern to those who need measurements at the extremities. These designs are close to orthogonal and may be used to estimate main and quadratic effects as well as all linear two-way interactions. Least-squares regression compensates for the nonorthogonality.
Central Composite Designs. These symmetrical, space-filling designs are flexible and efficient. Also known as Box-Wilson designs, these multidimensional cubes are face-centered and may be easily rotated by choice of appropriate factors. This easy rotation means that the predicted response may be estimated with equal variance regardless of the direction from the center of the design space. The only drawback in these designs is that the number of runs required for the larger designs rapidly becomes very large.
Designing an Experiment
The following brief outline of steps illustrate the design and interpretation of a response surface experiment.
Choose the Design Type. In immunology it is usually important to fully understand the contributions of each reagent to the final reaction. In such a case, the researcher would select a full factorial model in order to include each input variable. A fractional factorial design might be chosen if only a subset of the main variables were considered important.
A quadratic design is usually chosen for the initial response surface experiment, since by mathematical design and historical use they have been found the most effective for immunologic experiments. Such a design considers not only the main effects (in this case the individual reagents) but also the interactions (for example, does antigen x react in any way with antibody y) and the quadratic terms. These last are the squared terms and describe the curves or bends in the response surface that help fit the calculated surface to the data. No physical meaning is generally ascribed to the quadratic terms (what does "antigen squared" mean in the real world?).
The cubic, Box-Behnken, and central composite designs are somewhat specialized and are included here for completeness. Rarely are cubic and higher-order designs needed. By experience, quadratic designs have been demonstrated to accurately describe physical data sets to within generally accepted ranges. Furthermore, the mathematics are greatly simplified by restricting the model to first- and second-order terms, and the interpretation of individual factors and interactions is more meaningful.
Implement the Design. In the second step, the laboratory data are collected and observed for possible exclusion of outliers. Portions of the experiment may be repeated. The data are then analyzed in the design software.
Interpret the Results. A typical outcome would be as follows: At x concentration of antigen and y concentration of antibody, the largest reaction rate is observed. If the degree of certainty surrounding this rate is acceptable, the laboratory's standard operating procedures will be written to specify mixing these levels of antigen and antibody to ensure the stated reaction rate plus or minus system error.
Conclusion
This discussion by no means exhausts the types of designs available, both for screening and response surfaces. It is, however, a representative sample of the more commonly used design types. The example provided above demonstrates the relationship between the mathematical abstractions and the actual process being measured and manipulated.
Part 2 of this article (which will be posted here at the end of October, 1997) will further demonstrate the practical application of the principles outlined here. The screening and response surface designs for master lot testing for a serum protein will be explored in detail, and commercially available software packages will be described.
Bibliography
Atkinson AC, and Donev AN, Optimum Experimental Designs, Oxford, Clarendon Press, 1992.
Myers RH, and Montgomery DC, Response Surface Methodology: Process and Product Optimization Using Designed Experiments, New York, John Wiley, 1995.
Schmidt SR, and Launsby RG, Understanding Industrial Designed Experiments, 3rd ed, Colorado Springs, CO, Air Academy Press, 1992.
Wheeler B, ECHIP Reference Manual, Hockessin, DE, ECHIP, 1993.
John A. Wass is a mathematical analyst in the scientific support group at Abbott Laboratories (Abbott Park, IL).
