IVD Technology Magazine
IVDT Article Index

Focusing the front end of
IVD instrument development

Bill J. Wood and Lindsay T. Wert

For instrumentation projects, a conceptual architecture can be a key tool for communicating product visions and minimizing business risks.

In the field of IVD development, chemistry and engineering sometimes function as two disparate universes. At the front end of a new instrumentation projec—the initial steps that are taken when a project is first begun—any gap between these two disciplines can create uncertainty that can last right up to the time that the engineering team begins detailed design work.

Such a lack of clarity during the initial period of product definition can result in confusion over the requirements for an instrument, and invariably slows development of the product. Developers can overcome these problems, however, by ensuring that the goals established for the product accurately reflect the relationship between the system's chemistry and a suitable instrument architecture. When care is taken to ensure that this relationship is well understood at the front end of a project, benefits can include an early understanding of project risk, acceleration of instrument development, and shorter time to payback.

Depending on the goals for automation, the interlocked development of assay chemistry and related instrumentation can take many paths. An assay can be developed with different instruments being the target. Or an instrument might be designed to run many different assays. The technical success of such an instrument may depend on such capabilities as high throughput and a correspondingly tight coupling of the assay chemistry to the instrument.

IVD instrumentation at work: the Facs loader of the Facscalibur flow cytometry system by Becton Dickinson (San Jose). Photo Courtesy of Becton Dickinson Immunocytometry Systems

Regardless of the technical features that the instrument embodies, however, the completed product will only be considered successful from a business standpoint if it fulfills the single requirement of automating a diagnostic assay. This single criterion is more important than all others. It is more important than whether the designer has considered human factors, whether the development process complies with FDA's quality system regulation (QSR), whether the labeling is correct, or whether the packaging is attractive. Nevertheless, making the product fulfill its primary requirement should not prevent designers from also satisfying other criteria. The approach presented in this article supports QSR compliance by emphasizing the need for teamwork and a dialog that will lead to an early establishment of product requirements.

This article is concerned primarily with the steps we use to develop an instrument architecture that accurately automates one or more diagnostic assays. Secondarily, we focus on the steps that should be taken before making the business decision to proceed with such a program. Stated simply, this article examines the early aspects of engineering design and related business judgments.

Converting Visions into Reality

Most IVD programs begin with a vision statement and a statement of market needs. The earliest concept is commonly proposed in high-level terms, such as "We will develop a benchtop instrument that automates our proprietary cardiac marker assays." A market needs statement adds to this vision only enough requirements and detail to win approval to launch the engineering effort.

With these general statements in hand, the manufacturer can then proceed to bring together a development team for the project, usually a multidisciplinary group comprising representatives from marketing, field service, instrument engineering, assay development, and regulatory affairs. This team typically operates within a formalized development process whose first phase is devoted to defining the requirements for the instrument. It is often a surprise to new engineers that this phase includes more than that. In fact, it encompasses a number of key activities, such as risk analysis, safety analysis, assessment of human factors requirements, and formulation of project planning documents.

In our methodology, the early work-products of the project development team are embodied in several sets of documents (see Figure 1). We use the product requirements document as the repository for instrument requirements. Since the safety of the instrument is important, both for the patient and the operators, we use an initial risk analysis to identify hazards, the causes of those hazards, and any elements of design or processing that can be applied to mitigate risk. Finally, we use the system architecture to serve as the initial design document for the instrument. None of these work-products stands alone. In reality, they form an interrelated group of documents, with work on each often occurring in parallel.

Figure 1. Specification phase work-products. The work-products developed during the specification phase are coupled so that progress on one can influence the content of another. Key starting points for this phase are the market needs and product vision standards.

The Buried Activity

Buried within the dynamics that produce these interrelated work-products is another document known as the conceptual architecture. The conceptual architecture encompasses several products that result from working sessions between the chemistry and engineering teams. Because this document can serve as a focal point for communication between these teams, it is often one of the first items to be developed. Development of the conceptual architecture can be viewed as a subprocess that supports creation of the product requirements document, the initial risk analysis, and the system architecture.

A conceptual architecture is much more than a sketch on a napkin. It should include enough information to convey an understanding of the instrument's complexity and the work necessary to make the chemistry and instrument function together. The information provided by the conceptual architecture enables company management to generate sound estimates of project development and manufacturing costs at a time early enough to provide direction that can minimize wasted development effort. In turn, the conceptual architecture feeds the management process of assessing costs and project risks, ultimately leading to a go/no-go decision (see Figure 2).

Figure 2. Conceptual architecture subprocess placement. When instrumenting an IVD, the development of the conceptual architecture involves several working sessions to capture and map the chemistry to an automated concept.

The building blocks that make up the conceptual architecture—typically functional subsystems—are usually expressed at some common level of abstraction. Opening each building block discloses more subsystems. However, the layers of detail contained in the conceptual architecture are limited, with emphasis being given to the major subsystems. Where a fully refined instrument architecture would include a great deal of detail, down to the level of subsystem interfaces, a conceptual architecture usually employs a higher level of abstraction.

Assay Description

Since all in vitro diagnostic instruments are keyed to a single assay or family of assays, the logical starting point for a conceptual architecture is a definition of the target assay. In the case of a mature assay, this definition can be a very formal statement of the properties and processes related to the test. In cases where the assay is being developed simultaneously with its instrumentation, however, it is usually adequate for the chemistry and engineering teams to describe the test in general terms.

Table I. Comparison of the advantages and disadvantages of three styles of storyboards.

The first step in describing a new assay is to create a macroassay description—a high-level overview of the key steps needed to perform the test, from sampling through to analysis (see Figure 3). If the assay is at a point in development where many elements are still in flux, it is possible that input from feedback paths built into the development process will bring changes to the macroassay description. Because the macroassay description is such a general overview, however, it is more likely that no need for changes will be encountered, at least during development of the conceptual architecture. Once completed, the macroassay description feeds directly into the product requirements document.

To flesh out the macroassay description, it is essential that the development team take into account the maturity of the assay's chemistry. For instance, a mature assay will usually have been investigated sufficiently for the development team to understand all the relevant parameters, as well as the degrees of freedom that each automation step can take. But with immature assays, chemists may not have recorded the assay procedures fully, or the variability of some parameters may not be well understood.

Such limitations of understanding can make it difficult to adjust elements of the assay in ways that could simplify the instrument architecture. The procedures employed in a manual laboratory, for instance, may differ from those appropriate for an instrument. To correctly assess project risks, the project development team should have a complete understanding of the assay's chemistry and operational parameters, as well as of the effects that instrumented procedures may have on the assay.

For example, many assays are extremely sensitive to such processes as complex mixing or the heating and cooling performed in a reaction vessel. To determine the latitude that pertains to an assay, developers should explore the effects of such processing, including the rate at which the temperature of the reaction vessel is raised or lowered. Conducting such explorations while the conceptual architecture is being developed enables the project team to compose a set of working assumptions about what might be possible in the automated system.

The results of such explorations can also identify potential problem areas that will require closer investigation later in the development process, possibly using small automated assemblies that model the processes in question. Reserving detailed investigations for a later time enables the project team to complete the task of creating an understandable conceptual architecture without losing momentum. At the same time, identifying problem areas at this early stage is preferable to discovering them later, when unplanned investigations would exert pressure on the project schedule and increase costs above original estimates.

Developing the Process Storyboard

After the assay has been described, we use a storyboard as one of the pivotal tools for developing the conceptual architecture. The storyboard is essentially a communication tool. Its purpose is twofold: to ensure understanding of the assay among the instrument development group, and to communicate the instrument engineering to the assay development group. It presents the system in such a way as to promote two-way comprehension.

Figure 3. Macroassay description.

There are several different styles of process storyboards. Depending on the project requirements, they can be used individually or together to create a multidimensional view of the automated system (see Table I).

Developing a storyboard is not an ivory-tower activity. It goes hand in hand with instrument architecture development. Without involvement from the instrument engineers, and some knowledge of architecture, storyboards don't make sense. Additionally, storyboards must not be rigid. By developing them in a flexible format, evolutionary changes can be incorporated easily (see Figure 4).

Storyboards can be created for subsystems as simple as an automated testtube handler, or for complete systems as complex as a high-throughput multiple assay analyzer. The starting point is determined by the system's intended macroassay steps, such as hybridization, thermal ramp, sample extraction, amplification, or detection. Once these steps have been defined, the automated actions associated with each step must be isolated. Such actions may include fluid aspiration or dispensing, heating or cooling cycles, incubation, the application of forces such as magnetic flux, and so on. For each action, it is vital to define inputs, outputs, temperatures, mixing, timing, precision, and other quantifiable parameters.

To ensure that the storyboard accurately reflects reality, all of the functional areas represented on the project development team should be involved in creating it. This is especially important, because it is through the give-and-take process of developing the storyboard that the maturity and level of understanding behind the assay are revealed. This process is characterized by questions over such issues as how rapidly vessel temperature can be ramped, how much temperature overshoot the assay chemistry can withstand, or how precise dispensing operations must be. The challenges of mapping processes from the bench to the robotic environment of the instrument require the involvement of assay chemists and instrument architects alike.

If the assay has not been extensively investigated, part of the storyboard process should be devoted to formulating a list of assumptions relating to the flexibility of the assay, with a corresponding list of experiments required to test their validity. After several cycles of storyboarding, the team will arrive at an adjusted description of the assay. Together with the macroassay description, the adjusted assay description can be used to form sections of the product requirements document and to advance development of the conceptual architecture.

Linear graphic storyboard illustrating cuvettes, fluid delivery, aspiration, and other actions. Each cuvette represents a step in time. This type of storyboard can be used to communicate assay process in the device.

One important aspect of this work that management can monitor is the perceived maturity of the storyboard. Storyboarding can reveal whether the assay chemists understand the instrumentation process—and whether instrument engineers understand the assay process—well enough to minimize changes later in the development cycle. It can also show whether members of both groups can discuss the nuances of the process well enough to explain it. If either team is weak in these areas, the process is destined to be faulty and the probability for success is diminished. Thus, management should insist that each team achieve full understanding of the other's technology and methodology before beginning detailed design work. To support this, management should provide opportunities for each group to communicate what it knows, coupled with activities to double-check that the understandings are sound within the context of the assay and the instrument.

Instrumentation Factors

From the instrumentation side, two key streams of knowledge influence the development of the assay description. The first stream is embodied in the product vision statement, a nontechnical document establishing the concept for the instrument and typically drafted by a technical member of management. This essay establishes a view of the product from an executive perspective, and forms a starting point for all subsequent architectural thinking. In most companies, a product vision statement and a market needs statement are necessary to obtain the funding for the project. In addition, the product vision statement may describe the level of automation that will support items included in the market needs statement, such as walkaway operation, random access, high throughput, and so on. Although the product vision statement provides a starting point for the project, the development team may drift from this vision as it addresses the real issues involved in matching the assay to the machine.

Elements of risk assessment for an IVD instrumentation project. Developing an early statement of the project's risk makes use of the conceptual architecture as well as any changes that have been made to adjust the assay.

The other key stream of instrumentation engineering knowledge is delivered by way of the concept of dominant building blocks, which are market tested and accepted methods of accomplishing common instrumentation functions.¹ Such building blocks are derived from certain stable and well-understood subsystems of IVD instrument configurations that dominate the market. They generally represent some of the most efficient and reliable approaches to instrumentation, and exert a powerful influence on the methods used to map assays to automated platforms.

Dominant building blocks can be significant assets in moving a product to market quickly while at the same time reducing development risk. They eliminate the need to develop new techniques and equipment by providing well-defined ways of automating parts of an assay, as well as hardware components that can perform the desired tasks. Fortunately, most automated functions can make use of such dominant building blocks. Where this is not the case, however, engineers must invent new techniques to meet the needs of the market, thus increasing the business risks of the project.

Where do engineers start in developing assay automation? One good approach is to begin by identifying defining structures that can serve as cornerstones for the system architecture, and will subsequently determine how building-block elements must be adapted to fit into the system. To accomplish assay automation, the chemistry must come first, followed by such physical elements as mechanitronics and fluidics hardware, and finally by the process sequence, which is typically embodied in software.

Once these cornerstones have been established, "chaining" often occurs as part of the continued development of the instrument architecture. This means that the nature of one building block causes others to be adapted in certain ways. A decision to convey samples and reaction vessels on interleaved carousels, for instance, will determine the design of fluid-handling robots that can reach across and into the carousels.

Chaining is most common when an instrument's architecture is tightly coupled, as might be required for a highly efficient, high-throughput machine. With a more loosely coupled architecture, on the other hand, individual building blocks are less likely to be affected by changes and can use standardized interfaces to operate more or less independently of one another. Depending on the project, developers may deliberately adopt a loosely coupled architecture in order to reduce the scatter effect of changes in a major cornerstone (such as assay chemistry) or in specific steps of the system protocols.

Conceptual Architecture Components

The completed conceptual architecture package, then, is composed of the following elements:

* An assay matrix that includes the steps, volume requirements, CV requirements, and other major characteristics of the assay chemistry.

* One or more storyboards (see Table I).

* Hierarchical diagrams that illustrate the system architecture as a group of interconnected building blocks.

* A fluids worksheet that includes specific information on all fluids necessary to accomplish the chemistry (see Table II).

* A list of working assumptions developed by the project team, expressing its understanding of approaches that might work for the system, and summarizing the underlying bases of the storyboards and system architecture.

Although it is important to capture all this information in the conceptual architecture, rigorous documentation is not required. Instead, it is more important to record whatever concepts the members of the project team agree are potentially useful.

Once the conceptual architecture has been completed, it can be readily refined to become the system architecture—a formalized, reviewed, and QSR-compliant work-product (see Figure 2). This transformation can be accomplished by making two types of modifications to the conceptual architecture. First, detail must be added to specify the various types of interfaces that will exist between the major subsystems described in the conceptual architecture. Second, the conceptual architecture must be structured and packaged in such a way that a general nontechnical audience—beyond chemists and engineers—can understand the goals.

Understanding Project Business Risk

It is common for the continuation of an instrumentation project to hinge on a management go/no-go decision made soon after the development team has completed its assessment. Preparation for this subprocess is shown as the final block in Figure 2. It is essential that the entire management team focus its efforts on making this decision, to ensure that the business risks of the project are correctly assessed and minimized wherever possible (see Figure 5).

The starting points for management's risk assessment are the conceptual architecture and the adjusted assay description. These work-products—and a sense of how difficult they were to develop—contribute to management's understanding of the maturity of the project and the risks related to its schedule and performance goals. Without these documents, any modeling of the paths required for further project development would be, at best, only a guess. Management's risk assessment must also consider the regulatory and safety requirements of the project, elements that are essential but beyond the scope of this article.

The conceptual architecture also provides a key element that the management team needs to develop the first estimates of the cost of the manufactured instrument. Such cost estimates can be developed in two ways, both of which use information from the conceptual architecture as a base. First, the management team can develop a conceptual bill of materials (BOM) that will capture the costs of any dominant building blocks or other components specified in the conceptual architecture. This document should make use of the project development team's knowledge about the cost of dominant building blocks, the expertise of the company's manufacturing engineers, and pricing information provided by the purchasing support team.

Second, the management team can use the planning and scheduling documents created by the project development team to estimate the amount of time and effort that will be required to build and test the instrument. Unlike smaller therapeutic instruments, diagnostic instruments may require many hours of assembly, alignment, and testing. The cost of these operations can be a sizable portion of the overall manufacturing costs, and can strongly influence the total costs of the completed instrument. For these reasons, it is time wisely spent for the management team to accurately model the costs of the manufacturing process. Both the conceptual BOM and manufacturing effort model will be based on a set of assumptions that should be captured and listed together with the estimates of system costs.

As part of the management decision process, we compile all risk-related assessments into a project business risk package that includes feasibility information, development cost and schedule information, and device cost information. The package may also include information about other measures that may be early indicators of success in arriving on the market with the right chemistry, and the right instrument, at the right time. The management team uses this package as a basis for its decision on whether to proceed with the project, condensing its information into a statement of that decision and a set of recommendations for the project. The project risk package helps to speed the management decision process through the sharing of sound information, instead of mere speculation.

As part of the project business risk package, a fully developed conceptual architecture can be a valuable tool for evaluating the maturity of the assay chemistry and the type of engineering that will be needed to realize the project's proposed subsystems. By mapping this information on a two-dimensional grid (where the axes represent the maturity of understanding in each area), the relative risk of the project can be made readily apparent. Such a presentation format can clearly communicate the high risks involved in a project where knowledge of both assay chemistry and automation engineering are weak. In a similar way, such risk mapping can help to show whether increasing knowledge along one or both axes (e.g., by conducting benchtop investigations using proof-of-principle modules) would significantly reduce the business risk of the project.

Conclusion

At the uncertain front end of IVD development programs, business risk often hinges on the maturity of the assay chemistry and the building blocks that project engineers apply. Development of a conceptual architecture is a means of understanding the vision of an instrument that can realize the automation of an assay, thereby bridging the gap between these two disciplines. Not only does the conceptual architecture communicate the nature of that machine to those engaged in assay chemistry; if developed correctly, it can also bring the chemists' dream into the minds of the engineers.

Reference

1. RM Henderson and KB Clark, "Architectural Innovation: The Reconfiguration of Existing Project Technologies and the Failure of Established Firms," Administrative Science Quarterly 35 (1990): 9–30.

What chemists need to know about instrumentation

Having chemists think through the process of instrumenting an assay can be a great help when it comes time to design the necessary hardware. By anticipating the needs of their engineering colleagues, chemists can reduce the likelihood that impossible performance requirements will be expected for the completed instrument. Following are the top 10 instrument-related considerations that chemists should be aware of when designing an assay.

1. Reagent Variations. It is very difficult to design an instrument that will vary an assay's specified parameters in order to accommodate reagent variations. As a result, most instruments are not designed to perform any variation in assay parameters or to account for reagent variations in any way. High-throughput instruments are especially intolerant of reagent variations. Assays should be designed with these limitations in mind.

2. Parameter Variations. Before beginning instrument development, chemists should ensure that they understand the effects of variations in all the parameters specified for the assay, and that they know what variations the assay will (and will not) tolerate. The chemist should know, for example, what effect variations in fluid concentration or viscosity will have on the success of the assay.

3. Flexible Chemistry. Small changes in the chemistry of an assay can often facilitate instrument development. To produce the best collective result, chemists and instrumentation engineers must work together to understand and make use of
whatever flexibility is offered by the chemistry of the assay.

4. Tracing Errors. Sources of error exist in both assays and instruments. To ensure that the system will produce the desired results, both sources of error must be considered.

5. Limits of Flexibility. Assays that are performed manually have all of the training of the chemist behind them. Trained chemists can make minor adjustments in their techniques based on their perception of how well an assay is running. Instruments cannot make the same adjustments; they run the same assay the same way every time.

6. Manual Operations. Some pipette operations that can be performed easily by hand are very difficult or impossible to
perform with an instrument. Consistently dispensing volumes of less than 10 ml or tilting a pipette while dispensing, for instance, are both very difficult operations. Be aware of such limitations and avoid requiring them.

7. Test Conditions. Assays should be bench-tested under the conditions expected in the instrument. If the instrument will be required to refrigerate its onboard reagents, for instance, then the assay that is run on the bench should also use refrigerated reagents.

8. Assay Volumes. It is difficult to validate performance for a range of volumes, whereas it is much easier to validate several discrete volumes. When defining the range of fluid volumes to be used in an assay (e.g., for performing dilutions), chose several standard values rather than a continuum.

9. Fluid Properties. Matching the physical properties of the fluids used in an assay (e.g., their viscosity, pH, and surface
tension) will facilitate integration of the chemistry with the instrument.

10. Pipetting. In instruments, washable pipette tips are easier to deal with than are disposable tips.

Bill J. Wood is vice president for research and development, and Lindsay T. Wert is diagnostics systems director, at RELA Inc. (Boulder, CO).