Medical Device Link.

Originally Published IVD Technology November/December 2002

Detection Technologies

An alternative approach to optical imaging

An optics technology utilizing extended-depth imaging enhances slide screening and discovery applications.

Alan E. Baron and Vladislav V. Chumachenko

Recent innovations in sample handling, image capture, and data processing have greatly improved the utility of automated microscope imaging systems for slide scanning and high-throughput screening (HTS) applications. Despite such innovations, the design of the optics used in these systems has changed little in the past 50 years. Significant improvements in sample throughput and analytical accuracy would result from incorporating imaging optics that are at a level of sophistication comparable to other system components.

Automated microscope systems that digitize, analyze, and archive in vitro preparations can improve the efficiency of pathologists by removing some of the sample-imaging burden. Commercial systems of this type digitize slides at high resolutions and archive results for later examination. In drug discovery applications, high-throughput imaging systems acquire enormous amounts of image data for subsequent analysis. The need to determine the best focus every time an image is acquired becomes the principal factor limiting throughput of slide-digitizing and HTS systems.

An optical technology that increases the depth of field of microscope imaging systems also increases the speed and efficiency of automated slide-screening and discovery applications. This technology, called Wavefront Coding (CDM Optics Inc.; Boulder, CO), can eliminate the need to refocus between analysis locations in automated screening and discovery applications. Compared with systems using traditional optics, Wavefront Coded microscopy extends the depth of field by a factor of 10x or more, increasing throughput while minimizing the complexity of the system and sample preparation.

Wavefront Coding did not evolve out of the realm of traditional optics, but was the result of applying mathematical techniques used in radar to optical applications.¹ With Wavefront Coding, the imaging burden is shared between jointly optimized optical components and signal processing algorithms. This combination of optics design and signal processing provides a solution to the focus problems found in diverse microscopy applications. This article shows how extended-depth imaging benefits two different applications: brightfield analysis of a Pap smear slide and fluorescent imaging of a microtiter well.

Slide Scanning Focus Problems

While complete archiving of a typical cytological slide preparation requires the acquisition of hundreds of images, many more images must be acquired to determine the best focus at each location of interest. The time needed for moving the specimen stage, waiting for the stage to settle, and acquiring the focus accounts for more than 70% of the time required to image a slide. Using Wavefront Coding for extended-depth imaging reduces the delays and system complexities associated with focusing procedures.

Focusing difficulties in slide-scanning applications fall into two categories: intraimage and interimage. Intraimage misfocus occurs when the thickness of a specimen exceeds the depth of field of the imaging system. Since many commercial preparation methods are designed to minimize the distribution of cells in depth, intraimage misfocus typically occurs only with magnifications of 40x or greater.

Interimage misfocus presents a more significant problem that results from nonflat specimen slides, tilted sample stages, or variability in slide positioning mechanisms. Even though the vertical distribution of cells in a small area of a cytological slide preparation may be within the depth of field of a 20x objective, the distance from the cells to the microscope objective can change by more than 30 µm while translating 20,000 µm (2 cm) horizontally across the sample area. Such variations in distance exceed the depths of field of 10x/0.25 and 20x/0.40 objectives, which are approximately 8.5 µm and 5.8 µm, respectively. To compensate for such variations in distance from the objective, an imaging system must find the best focus for each location on a slide specimen.

Figure 1. A conventionally prepared Pap smear viewed with a traditional microscope (a–c) and with extended-depth imaging using Wavefront Coded microscopy (d–f). (click to enlarge)

To illustrate the problems associated with misfocus, portions of a conventionally prepared Pap smear slide were imaged using a Zeiss Axioplan-2 with a 20x/ 0.50NA objective (see Figure 1). The focus for the first location was set manually using a computer monitor to view image 1b. The specimen stage was moved to the left by 1.5 mm for acquiring image 1a, and then to the right by 11 mm for acquiring image 1c. The focus was left unchanged during the entire linear displacement.

In image 1b, the whole field of view is in clear focus. Image 1a is mostly in focus, but shows intraimage misfocus in some areas due to the thickness of the cell layer. Image 1c is 9.5 mm to the right of image 1b and is out of focus across the entire image. The interimage misfocus between images 1b and 1c results from a combination of the mechanical tolerances of the slide and stage mechanism, and could not be averted by using a thinner slide-preparation method. This example shows the need for readjusting focus at each imaging location in automated scanning applications.

Using the same 20x objective, this imaging sequence was repeated with Wavefront Coding technology integrated into the microscope. The focus position for the first extended-depth image, 1e, was again set manually, observing the image on a monitor. The next two images (1d and 1f) were taken at the same locations as before, with the focus left unchanged.

A comparison of the two image sequences demonstrates that Wavefront Coded microscopy extends the depth of field and produces clear images for the entire slide. Both the intraimage misfocus in image 1a and the interimage misfocus in image 1c have been eliminated. For example, cell nuclei that are not visible in image 1c show up clearly in image 1f. With the depth of field extended fourfold, Wavefront Coding could also have reduced the time required to image the entire slide.

Competing Technologies

Various technologies have been developed to minimize focus problems for microscopy applications. These technologies include laser scanning microscope systems, sophisticated but time-consuming refocusing procedures, optics that sacrifice resolution for greater depth of field, and image deblurring or deconvolution algorithms. None of these technologies mitigates focus problems with the simplicity and speed required for slide scanning and screening applications.

Laser scanning confocal microscopy is a powerful tool that constructs a three-dimensional image of a specimen with all image planes in clear focus. These systems, however, require complex instrumentation, acquire images in minutes instead of milliseconds, and are generally limited to fluorescent applications.

For fluorescent cellular assays, a series of images taken over a range of distances from the objective can be recombined using simple arithmetic to project the maximum intensity value of each pixel onto a composite image. This simple recombination suffers from reduced contrast due to stray light, and requires acquisition of several images at each location. More-sophisticated deconvolution algorithms which deblur a series of images have been widely reported in the literature. The downside is that in addition to requiring multiple image acquisition, the deconvolution process is iterative and may take from minutes to several hours.

Some commercially available automated imaging systems use proprietary models that attempt to predict focus at each image location. Even though this type of system gives a better first guess at the correct focus distance, it is still necessary to acquire several images to confirm and fine-tune for the best focus.² The primary speed-limiting step in most automated systems is the time required for the stage and focus system to settle after being repositioned. Since a vast majority of the images are taken only to determine focus, systems using Wavefront Coded optics could reduce acquisition time by 50–75%.

Wavefront Coded Optics Explained

Figure 2. Wavefront Coded optic CPM127-R60 with 20x/0.5 and 50x/0.8 Zeiss objectives for the Axioplan-2.

The Wavefront Coding technology extends the depth of field of a microscope objective and integrates into both the optical and signal-processing paths of an imaging system. In microscopes, the Wavefront Coded optic fits directly into the differential interference contrast (DIC) slot without any other modification to the optical system (see Figure 2). In the signal-processing path, an additional linear filtering step recovers the extended depth of field of the Wavefront Coded image.

In traditional microscopy, the primary function of the optics is to provide a sharply focused image of the specimen at the image detector. This requirement reduces the flexibility in other aspects of the system. A microscope may experience a reduction in depth of field from 8.5 µm to 1 µm when switching between 10x/0.25 and 40x/0.65 objectives. In traditional microscopy, higher magnifications result in greater resolution, but only at the cost of reduced depth of field.

Extended-depth microscopy using Wavefront Coded optics differs from traditional microscopy in several key aspects. Positioned near the back aperture of the microscope objective, the Wavefront Coded optic is an aspherical optical element that codes phase information into the image path. Because of this coding, the image directly formed at the detector never achieves good focus—to the eye the image appears blurred. However, a unique feature of the blurred image is that the blur is constant over a large range of focal distance. This blurred image can be processed to restore sharp focus over a larger depth of field when compared with the traditional microscope image.³

Figure 3. Traditional microscopy images of 0.5-mm pinholes at different focus positions (a–c). These images are termed point spread functions (PSF). Images d–f are intermediate PSF images using Wavefront Coded microscopy. Images g–i are filtered PSF images using Wavefront Coded microscopy. (click to enlarge)

Comparing a sequence of images taken with traditional and Wavefront Coded optics demonstrates the principles of extended-depth imaging (see Figure 3). Images of pinhole-like objects, called point spread functions (PSF), are commonly used to characterize the performance of microscope systems. The PSF image reveals how the objective misfocuses as an object is moved away from the point of best focus. In Figure 3, images a–c show three different images of a backlit 0.5-µm pinhole observed with a 40x/NA=1.3 oil immersion objective using a Zeiss Axioscope.

Taken at best focus, image 3a shows a sharp, resolution-limited image of the backlit pinhole, while images 3b and 3c are taken with the pinhole at increasing amounts of misfocus. As a result of the misfocus, the light in images 3b and 3c has been spread over a large area and no longer resembles the original pinhole object. The intensity of the out-of-focus images has been normalized by increasing exposure with misfocus. If image 3c were taken with the same exposure as image 3a, it would appear almost completely dark.

Images d–f show that the PSFs taken with a Wavefront Coded objective differ in appearance from images a–c. With a Wavefront Coded optic in place, the best-focus image (image 3d) no longer resembles the corresponding traditional best-focus PSF. The shape of the Wavefront Coded optic distributes most of the light energy coming from the pinhole into two orthogonal legs emanating from a bright spot.

More importantly, as the pinhole is moved away from the best-focus position, the Wavefront Coded PSF does not change as dramatically as it did in the traditional system. Images d–f that show the blur introduced by Wavefront Coding are called intermediate images. The invariance in the intermediate images as a function of distance is a direct consequence of Wavefront Coded imaging.

Though the PSF images d–f are constant over a large range of focus compared with the traditional images, these intermediate images do not resemble the original pinhole object. A final processing step must be applied to the intermediate images to recover those that look like a traditional system's best-focus PSF. Images g–i show the Wavefront Coded images after processing. These images appear sharp even at the largest misfocus, where the traditional PSF images are badly out of focus. It is important to note that the signal processing applied to the PSF is object independent. The same algorithm recovers a resolution-limited pinhole image as well as it would recover a monochrome image of a fluorescent cell nucleus, or a color brightfield image of a Pap smear preparation.

Signal Processing and System Calibration

Since the PSFs of the intermediate images are invariant over a wide range of misfocus, the same signal processing can be applied to all intermediate images. This processing step consists of a one-time (noniterative) convolution of a digital filter with the intermediate image. On a Pentium III–class computer, it takes less than 100 milliseconds to process a 1280 x 1024–pixel intermediate image. With such short processing times, Wavefront Coding provides an alternative not offered by other image-deblurring techniques that use iterative algorithms requiring minutes to hours for completion on 512 x 512 images.⁴

The calibration process for an imaging system generates a filter kernel that is used with a specific optical configuration. In a microscope system, the filter kernel is determined by the optics' shape, the specific objective type and magnification, the intermediate relay magnification, and the detector pixel pitch. Once an imaging system with a certain optical configuration is calibrated, the filter kernel solution can be used for all platforms with this same configuration. Determining the filter kernel begins with acquiring several PSF images and submitting them to proprietary software, which settles on a filter kernel that gives the best-quality filtered images.

There are several ways to incorporate the processing algorithm into the image processing path. CDM Optics Inc. (Boulder, CO) has developed dynamic link libraries for Windows 2000 which can be called by the system's control software. Filter kernels can also be formatted for use with the image convolution function in MetaMorph by Molecular Devices Corp. (Sunnyvale, CA), and in IP Lab by Scanalytics Inc. (Fairfax, VA). The processing algorithm has also been implemented in digital hardware that enables processing of 640 x 480–pixel images at 110 frames per second. Since the intermediate image is immediately intercepted after being digitized, the Wavefront Coded processing is transparent to the end-user, who receives an in-focus image that can be further analyzed and archived.

HTS Fluorescence Assay Application

Another characteristic of extended focus with Wavefront Coded optics is its applicability to a wide range of microscope imaging modes and applications, such as brightfield, darkfield, fluorescence, Hoffman modulation contrast, DIC, and polarization imaging modalities. The following example demonstrates that the utility of Wavefront Coded microscopy is not limited to brightfield imaging of tissues and cell preparations, but can be used to improve even the most sophisticated fluorescence assays.

Figure 4. Traditional microscopy (a–c) and Wavefront Coded microscopy (d–f) images of Texas Red fluorescently labeled beads adhered to the well of a microtiter plate. (click to enlarge)

The microtiter plates used in many discovery applications present misfocus problems similar to those encountered in slide imaging. A commercially available plastic-bottom 96-well microplate varies in height by as much as 30 µm from the center to the edge of a well over a horizontal displacement of 3000 µm, or half the 6-mm diameter of a single well. A three-image sequence shows 1 µm-fluorescent beads adhered to the bottom of a microtiter well using an inverted microscope with a 40x/0.6 ELWD objective and a Hamamatsu Photonics Orca 2 CCD camera (see Figure 4). Image 4a was taken near the edge of the well, with the focus set manually. The microscope stage was then moved toward the center of the well in two steps, with images acquired at each location without adjusting the focus position. Images 4b and 4c show significant misfocus, resulting in reduced brightness as the light from the bead is spread over an increasingly larger area.

This experiment was repeated using the same optical configuration, except that a Wavefront Coded optic was inserted into the DIC slot on the nosepiece. In images d–f, the processed Wavefront Coded images show little or no misfocus. Consider the number of steps required if an HTS user intended to perform a discrete cell assay within this microwell at these three locations. Assuming that focus verification would require at least four images at each point, then 15 steps of acquisition and stage movement are required for only three different locations. With Wavefront Coded optics, only the first focusing step would be necessary, requiring seven steps for image acquisition and stage repositioning.

Wavefront Coding Microscopy Limitations

Wavefront Coded optics offer alternatives in situations in which intra- or interimage misfocus restricts the throughput and efficiency of an imaging system. However, certain imaging applications do not benefit from extended-depth imaging. Some image analyses optically section a thick specimen, taking advantage of the limited depth of field of a high-resolution objective. Depending on the objective's numerical aperture, the depth of field may be as small as 0.5 µm, thereby enabling visualization of extremely thin sections. In this case, Wavefront Coded optics would not be suited because it brings the entire specimen volume into clear focus.

All image-processing algorithms tend to amplify existing random background noise, and the processing for Wavefront Coded optics is not an exception. However, there are several strategies for minimizing noise amplification in extended-focus images. The dominant source of random noise in the system is the camera's electronic noise. Wavefront Coded optics can be adapted to nearly any commercially available camera so that the selection of a device with an appropriately high signal-to-noise ratio is not a limitation.

In addition, the amplification of random noise increases proportionally with the depth-of-field extension. The Wavefront Coded optic configuration determines the amount of depth-of-field increase and should be chosen to provide just enough increase for the particular application. An excess extension of the depth of field adds noise that could otherwise be avoided.

Conclusion

Extended-depth imaging with Wavefront Coded optics provides a tool to enhance automated microscopy imaging applications. While this technology represents a radical departure in the way optics and signal processing are combined, the integration into automated systems requires little modification to the optical system and only adds a manageable additional step to the signal processing and data analysis pipeline. After signal processing, the extended-depth images can be further analyzed and archived the same way as in traditional microscope systems. By incorporating Wavefront Coded optics, significant time savings can be realized by eliminating or reducing the need for focus readjustments during the digitization of a sample. As more imaging assay techniques become available, Wavefront Coding will allow the acquisition of greater amounts of data in the same or less time when compared with systems using traditional microscope optics.

References

1. ER Dowski et al., "Extended Depth of Field through Wavefront Coding," Applied Optics 34 (1995): 1859–1866.

2. JM Geusebroek et al., "Robust Autofocusing in Microscopy," Cytometry 31 (2000): 1–9.

3. S Tucker et al., "Extended Depth of Field and Aberration Control for Inexpensive Digital Microscope Systems," Optics Express 4 (1999): 467–474.

4. AS Carasso, "Linear and Nonlinear Image Deblurring: a Documented Study," SIAM Journal of Numerical Analysis 36, no. 6 (1999): 1659–1689.

Alan E. Baron, PhD, is director of business development and Vladislav V. Chumachenko is an optical engineer at CDM Optics Inc. (Boulder, CO). The authors can be reached at alanb@cdm-optics.com and vladc@cdm-optics.com, respectively.

Copyright ©2002 IVD Technology