Medical Device Link.

Originally Published MEM Fall 2003

MEDICAL IMAGING

Balancing high performance, high reliability, and cost-effectiveness is possible with new switched-fabric-based technology.

Justin Moll and Stephen Christo

Medical imaging systems are requiring higher bandwidth. With the emphasis on real-time performance and higher resolutions, the amount of data processing needed may soon reach staggering levels. Using input sensors to receive large amounts of information along with digital signal processors (DSPs) to turn that analog input into digital data, medical systems are leaning toward backplane-based chassis with high performance. Traditionally, many systems use standards-based architectures like CompactPCI bus or VMEbus. But with backplane performance hitting the limitations of standard architectures, medical industry system designers are looking to new switched-fabric technologies that offer high performance and high reliability at a reasonable cost.

Medical imaging systems—such as magnetic resonance imaging (MRI), computed tomography (CT) scanning, and positron emission tomography (PET) scanning—are the most in need of higher performance. The intrachassis traffic between the processor boards, communications boards, display, sensor device, and storage is vast. For real-time information and clearer, higher-resolution images, the data need to travel at high speeds without errors. Not only is the data rate important, but the processing must be reliable. With redundancy and inherent reliability, the system can be highly available, with up to 99.999% uptime.

New Technology

A move to a switched-fabric-based technology is a natural evolution for medical imaging systems. Switched-fabric technology inherently provides some key benefits such as design flexibility, physical and bandwidth scalability, reliability, and quality of service. In bus technologies like PCI, only one device at a time has access to the bus. Each device must request the bus, and an arbitration algorithm is used to grant requests. In contrast, the duplex nature of a point-to-point switched interconnect allows each device to transmit and receive simultaneously. Through the building of systems with series of end-points and switches, a diverse and a flexible array of system topologies can be created. As more connections are added to the system, the total bandwidth of the system increases. This networked approach also provides significant flexibility in the system design topologies that can be created.

With a serial switched architecture, both physical and bandwidth scaling are realizable. Today's buses are typically limited to 12–20 in. on a backplane and 1–2 ft over expensive and unreliable ribbon cables. Using serial physical layer technology, these distance limitations can be eliminated. In some medical equipment, such as ultrasound machines, the storage and display systems are typically separate from the analog input and image-processing arrays. With a serial interconnect, developers are not limited to placing these segments of the system adjacent to each other.

To scale the bandwidth of a bus-based system, either the bus is widened or the frequency is increased. Both methods have their drawbacks. Increasing the width requires additional routing layers on backplanes and boards, subsequently increasing the total cost of the system. Increasing the frequency on synchronous buses tightens skew and routing requirements, limiting the number of devices supported and increasing reliability concerns. By implementing a serial switched interconnect, increasing the overall bandwidth of a system is as simple as adding another endpoint to the topology.

In addition to scalability, switched interconnects provide a dramatic increase in reliability compared with bus-based designs. In contrast to buses, in which an errant endpoint can bring down the entire bus, the point-to-point nature of switched interconnects isolates faults to a single endpoint. The fault at an endpoint could be handled in several different ways. The most basic response would be to remove the faulty field replaceable units (FRU) and hot-swap them with replacements. Point-to-point connections are inherently friendly to device insertion and removal. Mean time to repair is minimal.

However, the inaccessibility of a technician could result in costly equipment downtimes. Another method would take advantage of the redundancy provided by switched interconnects. For example, a CT scan system might have several central processing units (cpus) running different algorithms for processing images. If one of these cpu blades fails in a system, the traffic targeted at that blade could be automatically rerouted to another cpu card using the redundant routing capability of switched interconnects. If the redundant cpu is a standby processor, the system will continue to operate with no performance degradation after a downtime measured in seconds instead of hours. If the redundant cpu is actively running another algorithm, it could expand its processes and run a second algorithm. In this case, the system would run at a reduced performance level until a replacement cpu is installed.

Quality of service (QoS) is another benefit that switched interconnects have over bus-based systems. In a bus-based system, the best a designer can do to prioritize a specific data stream is to develop an arbitration algorithm that provides preferential treatment to the master generating the high-priority data. Using a switched interconnect like StarFabric, specific data streams can be tagged with a higher priority class of service. In platforms where different types of data converge, this capability greatly enhances system performance.

Switched Fabric

StarFabric is a serial switched interconnect with a physical layer consisting of low-cost, high-performance 622-Mb/sec low-voltage differential signals (LVDS). Four 622-Mb/sec differential pairs form a link with an aggregate bandwidth of 2.5 Gb/sec in each direction. Each implements both 8B/10B and CRC encoding to protect against bit errors in the data. If either an 8B/10B or a CRC error is detected at the receiver, the data will automatically be retransmitted by the hardware. With the 8B/10B encoding, the actual data bandwidth supported by a link is 2.0 Gb/sec, or 250 Mbyte/sec. The serial links are hot-plug capable and provide point-to-point connections between chips on a card, across a backplane, and between racks in a room. Chassis-to-chassis interconnects are possible with standard RJ45 connectors and CAT5 cable, eliminating any requirement for expensive high-speed connectors or cables.

Switched-fabric technology enables three types of routing methods: address routing, path routing, and redundant routing. Address routing provides 100% compatibility with PCI drivers, BIOS, configuration, etc. Path routing provides advanced features such as QoS, which enable data, voice, and video to be carried in a single interconnect. A typical system would use the asynchronous QoS for data traffic and an isochronous line for video. Multicast allows single initiators to broadcast messages to multiple destinations. Other high-availability features include error detection, correction, notification, and isolation. Redundant routing is supported with automatic fail-over to an alternate path when a connection is removed.

Figure 1. Typical backplane used in MRI systems. (click to enlarge)

PCI (and particularly CompactPCI) are often used in MRI systems (see Figure 1). The CompactPCI bus provides a rugged, hot-swappable platform for high-reliability medical systems. But CompactPCI's bandwidth is running out of steam. When running at 66-MHz clock rates and 64-bit bus widths, CompactPCI's bandwidth is 528 Mbyte/sec but the backplane is limited to five slots. For 33-MHz clock rates, the backplane can extend to eight slots (without bridging).

Figure 2. A Standard CompactPCI system compared with a switched-fabric system. (click to enlarge)

MRI, CT scan, and PET scan systems have many processors, A/D and D/A, data acquisition, and image-processing cards, and therefore require higher slot counts along with the higher bandwidths. The PICMG 2.17 specification based on switched fabric extends the performance of CompactPCI. Consider the two systems shown in Figure 2. The CompactPCI system uses a 64-bit, 66-MHz bus connecting five image-processing cards. In this system, the total bandwidth shared between all five cards is 528 Mbyte/sec. In the PICMG 2.17 system, however, each card could be transmitting and receiving at the full bandwidth supported by a serial link (250 Mbyte/sec), thereby saturating its local 64-bit, 66-MHz PCI bus (~500 Mbyte/sec) with the two-way traffic. Assuming a fully distributed system, the total system bandwidth increases to 2.5 Gbyte/sec or approximately five times more than bus-based architecture (see Figure 3).

Figure 3. The diagram above shows StarFabric as the serial interconnect between the cpu, I/O, DSP, storage, and more in a medical imaging system. (click to enlarge)

In a PICMG 2.17 platform, StarFabric traffic is run across P3 (P4 is reserved for H.110 and much of P5 allows user I/O). It is also focused on providing this functionality in an easily adoptable way by not requiring exotic system design in terms of power or signal integrity, and by allowing use of standard cabling and connector technology. Its cost structure is in line with traditional bridging technology. PICMG 2.17 products are now readily available. The backplanes can be developed in relatively low layer counts, such as a 12-layer controlled-impedance stripline design.

The CompactPCI busing on P1 and P2 and the H.110 bus on P4 can be routed in eight layers. Some of the StarFabric links can be routed on the same eight layers and the remainder on the other four layers. The differential pairs are routed as close together as possible and kept on the same layer. The outside layers are grounded for EMI protection and suppression. The signal layers are alternated with power or ground layers for controlled impedance and to minimize crosstalk. Vias are not used on signal traces because they disrupt the impedance of the trace. High- and low-frequency decoupling capacitors are distributed generously across the backplane. Power and ground planes typically use 2 oz of copper for power distribution.

The PICMG 2.17 specification defines two types of topologies, centralized and distributed, for integrating the switched fabric onto a CompactPCI backplane. A centralized topology uses a dedicated fabric board where all of the node board traffic is aggregated. Each line connecting a node board to a fabric board consists of a StarFabric link with 2.0 Gb/sec of bandwidth in each direction. By adding another fabric board with connections to a second link in each node board, a fully redundant system is possible. The redundant link can increase bandwidth to each node board or increase the reliability of the system.

Although a centralized topology provides an effective and scalable solution for a heavily loaded chassis, it may not be necessary in systems with a limited amount of slots. By implementing a distributed topology, dedicated fabric slots that only perform switching are no longer required. Instead, the switching functionality is incorporated into each line card. This topology is an efficient implementation for small-scale systems and provides a path for incremental growth as system requirements increase. For either topology, the CompactPCI form factor provides many high-availability features such as hot insertion and extraction and intelligent platform management interface (IPMI). The serial links of the fabric provide a more robust physical layer than CompactPCI for hot insertion and extraction of boards. In-band management complements the capabilities of IPMI system management.

Unlike other switched-fabric technologies, StarFabric supports several traffic classes including asynchronous, isochronous, multicast, and high priority. Asynchronous traffic is traditional data-oriented traffic, with large bandwidth requirements but without real-time delivery requirements. Control and signaling traffic are typically asynchronous. Isochronous traffic, including voice and video, requires deterministic real-time delivery. Through the use of these traffic classes, this new technology is ideally suited for applications such as MRI and other medical imaging systems in which data and video are converging.

Application Example

Figure 4. This example shows an ultrasound system incorporating StarFabric. Within a standard PICMG 2.17–compliant chassis, the image-processing levels and reliability of the system greatly increase. (click to enlarge)

In many medical applications, latency and throughput are important requirements. In an ultrasound system, latency is critical because the technician uses the real-time image to properly position the transducer. Latency is less critical in CT scan equipment, although it is still an important factor. Both systems require the performance to scale as processing elements are added to a system. If all of the processing boards are contending for a common bus, performance could actually degrade as the number of processing nodes increases. This new technology addresses these concerns by providing both physical and bandwidth scaling in a low-latency interconnect. Consider the ultrasound system shown in Figure 4.

In today's systems, the control cpu is typically connected to the input/output (I/O) and image processing boards with an expensive custom cable that carries all the PCI bus signals. Due to skew and signal integrity issues, these cables become unreliable if they exceed 1 or 2 ft in length. By replacing the cable with StarFabric serial links, the physical limitations are eliminated because the fabric can scale up to 10 m using standard CAT5 cable. In addition, it provides backward compatibility to legacy PCI while maintaining a very low latency (<1 µs). For this simple PCI expansion case, the new fabric enables developers to reduce system costs and improve system reliability without changing the current software.

Inside the PICMG 2.17 CompactPCI chassis, the boards are interconnected by StarFabric links using a distributed or mesh architecture. As image-processing requirements increase, DSP boards can be added to the system, and the overall bandwidth will scale accordingly. In addition, the multicast capability enables the I/O card to broadcast data to each of the DSP cards, which could then perform distinct operations. Less than a few microseconds are added to system latency. The additional latency would be orders of magnitude greater using another interconnect such as Ethernet.

Compatible Hardware–Chassis Considerations

Figure 5. The PICMG 2.17–compliant chassis from Elma Electronics shows a StarFabric backplane in a 12U unit with redundant power supplies, fan trays, and system management.

In medical systems, the chassis design is an important consideration. Addressing issues such as electromagnetic compatibility (EMC) and electromagnetic interference (EMI), ruggedness and a flexible design are critical for achieving high performance at a reasonable cost. PICMG 2.17–compliant chassis have been developed in a 19-in. rack-mount EMC version ideally suited for medical applications. The chassis in Figure 5 accommodates up to a 21-slot backplane with rear I/O capability. It employs a push-pull airflow technique using three individually removable plug-in fan trays below the cards, with 90-cfm tube-axial intake fans and dual radial blowers above the cards for exhaust.

The intake air is filtered using Bellcore-compliant foam air filters that are easily removable. Because the architecture is a migration path from traditional CompactPCI-based bus structure to a switched-fabric backplane architecture, existing packaging concepts can be used. Similar packaging solutions in areas such as EMI, cooling, and shock and vibration can be incorporated. For example, shielding with EMC gaskets and beryllium copper (BeCu) fingers or contact strips will help the chassis maintain electrical continuity between mating of metallic surfaces (panels, covers, etc.). Conducted emissions can be addressed by incorporating high-performance EMI line filters. Also, using the IEEE 1011.10 specifications makes EMI containment on the front panel and cards easier to achieve.

The cooling requirements for medical systems can be adequately met with advanced airflow techniques such as the slot air baffle and air plenums. Compact radial blowers or backward-curved impellers have proven effective in dissipating heat buildup under high static pressure. Employing fan-monitoring alarms is highly encouraged due to the mission-critical nature of medical applications.

Incorporating features like hot swap and redundant fan trays plays a major role in meeting high availability requirements. Therefore, the chassis accommodate pluggable fan trays, power supplies, and system management and monitoring. The use of 3.3, 5, and 12 V makes it possible to use readily available power-supply solutions. The PICMG 2.17 power-supply interface specification defines a 47-pin positronic connector for pluggable supplies. Power supplies are load sharing by using internal O-ring diodes. Power supplies monitor the health of the voltages on the backplane through either third-wire or droop-current-sharing methods. Current droop regulates to within 10–15%, and third-wire regulates to 5%.

Conclusion

Bus-based architectures are running out of bandwidth for today's medical imaging solutions. The migration to switched-fabric systems is the natural evolution path. It is important that a switched-fabric architecture already have all of the necessary hardware and software components. A switched-fabric system should also be compatible with PCI-based systems, so that much of a manufacturer's previous investment in the system can be preserved.

Justin Moll is marketing manager for Bustronic (Fremont, CA). He can be reached at 510-490-7388 or via e-mail at jmoll@bustronic.com. Stephen Christo is product marketing manager for Stargen Inc. (Marlborough, MA). He can be reached at 508-786-9950 or via e-mail at christo@stargen.com. For more information go to http://www.starfabric.org or to http://www.nextgenbackplanes.com.

Copyright ©2003 Medical Electronics Manufacturing