August 22, 2005 -- The world is moving from parallel to serial in order to meet the increasing I/O bandwidth requirements that are now a reality thanks to digital convergence. One has only to explore the new standards that are being deployed, in development, or being proposed. PCI Express, Serial RapidIO, and Infiniband are just a few of the serial standards that promise to solve the bandwidth challenge. One of the driving forces behind adoption of high-speed serial is the field programmable gate array (FPGA).

The FPGA is often in the center of designs performing such functions as high-speed digital signal processing, complex embedded processing, or system data aggregation. The ability to feed this juggernaut of compute power becomes daunting. While the strategy of replacing several hundred high parallel data lines (that must all arrive synchronously to a system clock and each other) with a relatively small number of very high-speed differential serial lines sounds appealing, in reality you are replacing one challenge for another. High-speed serial links that run from 622Mbps up to 10Gbps require advanced thought and careful board layout. From a debug and verification perspective, the challenge becomes more pronounced when you include the many settings available in high-speed serial transceiver technology embedded in FPGAs.

The traditional equipment available to test this technology can be expensive and is often ill-suited or more than what design engineers need to address the unique challenges faced by FPGA designers. Fortunately, there is a way to efficiently test designs that use this new technology; the FPGA itself. This article compares the system setup, supported capabilities, advanced features, and cost of two approaches for testing an FPGA design. One approach is a standalone Bit Error Ratio Tester; the second is a Bit Error Ratio Tester integrated within the FPGA.

But how do you get started? You need a solution that will allow you to choose the right technology, implement your design, and then verify and debug it.

Understanding the design stages of a high-speed serial design

What defines a working high-speed serial channel? This is a question that depends upon the design phase. For simplification, we will define the following three stages of a high-speed serial design implemented within an FPGA:

Technology evaluation: The design phase in which the technology is being evaluated to determine if it will function within a given set of conditions. This is prior to having a real world design. At this point, you simply want to know if the high-speed serial technology will function within a known set of conditions and whether the technology includes sufficient control and adjustment to compensate for real world conditions. During this phase of the design, you are validating the assertions made by the makers of the high speed serial technology. Design validation and margin analysis: The design phase in which you are exploring board layout, high-speed serial I/O settings, and operation of the high-speed serial channels with the rest of the system design. During this phase of the design, you may have determined the high-speed serial protocol and want to determine how much margin exists under the operational conditions. System operation: The phase at which the design is up and running, and the high-speed serial IO channels are operational within an overall system. You now want to determine how much margin exists in the design using mission data.

While the definition of a working high-speed serial channel will vary during each of these stages, fundamentally it is the ability to accurately transmit data over the channel at a given rate over a given period of time. One way to measure this is to determine the channels Bit Error Ratio, or BER. A Bit Error Ratio is the ratio of number of errors to the total number of bits transmitted.

There are other, more advanced methods to measure and define whether a high-speed serial channel is working; Protocol analyzers and high-speed sampling scopes for example are just a couple of the options. For the purpose of this article, we are going to focus on the first phase, technology evaluation. In this phase, a BER measurement is a good option for obtaining the information and confidence you need to move forward in your design.

How to measure the BER

Bit Error Ration Testers (BERT) exist to measure the BER of a high-speed serial link. BERT equipment works by driving a specific test pattern, most commonly a Pseudo Random Bit Stream (PRBS) into the device under test (DUT). The data is received and de-serialized within the DUT high-speed serial transceiver technology and then re-transmitted over the high-speed serial link to BERT equipment. As long as the pattern of data transmitted is known, a comparison between transmitted and received data can be made to determine if any errors occurred (see Figure 1). During the technology evaluation phase of a design, it is best to limit testing to a single device with the transceivers operating in what is referred to as loop-back mode.

Figure 1. How Bit Error Ratio is measured.

What determines a "good link?"

Telecommunication protocols require a BER of 10^-10 using long PRBS of 32 bits or greater. Alternatively data communication protocols like Fiber Channel and Ethernet require a BER of 10^-12 up to 10^-16 using shorter PRBS patterns. In order to guarantee that these BER levels are achieved, it is necessary to transmit more than the number of bits required within a given pattern to satisfy the probability of an error occurring.

As an example, consider a high-speed serial channel operating at 3.125Gbps. This channel can transmit 10¹² (1000 Gbits) in 5 minutes and 20 seconds. If no errors occur, then it could be assumed that a BER of 10^-12 has been achieved? This would be akin to manufacturing and testing one device and based upon this assume all subsequent devices are going to work. In reality, you would test multiple devices over a given period of time to determine the effectiveness of a given process. The same is true for determining BER. The longer the channel runs and successfully transmits bits without errors, the more assured the user is that the system will function properly under real-world conditions. Typically during the evaluation phase, the system is run until a certain number of errors are recorded and then the effective BER is determined from this datapoint.

What causes a link to fail?

High-speed serial links typically fail due to jitter, which can be categorized into two basic categories: random jitter and deterministic jitter. Random jitter is caused by any number of factors such as crosstalk, power supply noise, poor synchronization, etc. Deterministic jitter is predictable and occurs when a known event causes the system to misinterpret the binary value of a transmitted bit. Total jitter, the combination of deterministic and random jitter, causes errors in the system. If the system employs some kind of error correction mechanism, then infrequent errors can be detected and corrected. In systems where data can be retransmitted, error detection is typically enough. As high-speed serial channels replace parallel interfaces on backplanes and interconnects, error correction or error detection is often not available. In the technology evaluation phase of a design, it is important to obtain a BER that is well understood and provides sufficient margin beyond what the system requires.

Understanding Bit Error Ratio testers

Several of the major test and measurement vendors provide standalone BERT equipment with a variety of options. While the individual features may vary between equipment offerings from different vendors, one thing is the same; this equipment can be expensive and, unlike more common test and measurement equipment that support a variety of test capabilities, it generally only provides BER testing capability.

A BERT solution includes a pattern generator, used to generate the specific test patterns used to drive the device under test. It is necessary for the pattern generator to accommodate a wide variety of patterns to address telecommunication systems as well as data communication systems, both of which can be implemented in a FPGA.

A BERT solution also includes an error detector which performs the comparison between the patterns transmitted and the patterns received. In order to detect errors between what was transmitted and what was received, the error detector contains pattern checker circuitry that either generates the same pattern as the pattern generator, receives the same pattern that is transmitted, or mathematically determines the pattern based upon information about the pattern being generated. To remain synchronous to the pattern generator, the error detector receives a reference clock from the pattern generator. Often these components are integrated into a single piece of equipment, in some cases multiple components are combined to create a BERT solution.

Exploring BERT options

Consider for a moment that you are in the technology evaluation phase of a high-speed serial design and need to evaluate high-speed serial technology embedded into the latest FPGA technology. You could purchase BERT equipment, find equipment within your company that is not being used, or rent the equipment. Alternatively, because you are using an FPGA, you could implement this functionality in the FPGA itself. Building the pattern generator would be straightforward. The patterns are well defined and implementing them in logic would not be difficult. The error detection logic is straightforward as well given that the pattern generator is in the same FPGA. It would be a simple comparator between the pattern generator and the data received. A simple counter could be used to count transmitted data and errors that occur; 64-bits would be more than adequate.

The difficulty is in how to capture and display the results. Fortunately, a solution exists. On-chip debug has become a valuable tool in the FPGA designer's arsenal to tackle debug and verification. The same principles that allow communication via JTAG with debug and verification cores could be used to interact with the pattern generator and error detection cores. Figure 2 illustrates such a system. To understand the technical aspects and potential benefits of such a solution, we'll compare a standalone BERT solution against an integrated BERT solution implemented within an FPGA used to determine BER in the technology evaluation phase of a high-speed serial design.

Figure 2. Integrated BERT solution using FPGA logic.

Standalone BERT vs. integrated BERT

For the purposes of this comparison, the integrated BERT solution will be defined to include the following:

Pattern generator Error detector High-speed serial transceiver control logic

Communication with this BERT core will occur over JTAG, leveraging the same communication protocols established for on-chip debug and verification of the FPGA. The Standalone BERT solution will include integrated pattern generator and error detection logic and will support high speed serial rates greater than 10Gbps.

Setup

In both cases, the FPGA setup is very similar. Both solutions assume an evaluation board designed to evaluate the FPGA. In addition to the necessary reference voltages and grounds to power the FPGA, the evaluation board requirements include clock sources for the high-speed serial transceivers and SMA connectors to interface to the transceivers. The configuration of the FPGA will vary slightly; the integrated BERT solution will consume logic and BlockRAM to implement the BERT core itself and will include configure logic to setup the high-speed serial transceivers. The standalone BERT solution will only require configure logic for the high-speed serial transceivers. Ordinarily consuming additional logic and BlockRAM within a design could be problematic, but in this case the amount of logic consumed by the integrated BERT solution is a non-issue as there is no design implemented within the FPGA at this stage of evaluation.

The stand alone BERT solution can utilize either a configuration prom to configure the FPGA or a lab computer system to configure the FPGA via JTAG. The integrated BERT solution will require configuration via JTAG using a lab computer system. This lab computer system will communicate with the BERT core placed in the design as well. Figures 3 and 4 show the setups for the two BERT solutions.

Figure 3. Setup for standalone BERT testing.

Figure 4. Setup for integrated BERT testing.

Supported capabilities

Capabilities are defined as the ability to drive specific patterns into the FPGA, run at the variable rates available in the FPGA transceiver technology, and finally fully test all the high-speed serial transceivers that are available in an FPGA.

Patterns

To evaluate high-speed serial transceivers, it is necessary to drive specific test patterns that best match the anticipated data used within the final system. Both the standalone and integrated BERT solution support patterns that meet the American National Standards Institute (ANSI), Alliance for Telecommunications Industry Solutions (ATIS), International Telecommunications Union-Telecommunication Standardization Sector (ITU-T), and IEEE 1007-1991 requirements. These PRBS patterns include: 2^7-1, 2^9-1, 2^11-1, 2^15-1, 2^20-1, 2^23-1, 2^29-1, 2^31-1 inverted and non-inverted, with bit word lengths up to 40-bits. Both solutions support K28.5 used to test for Fiber Channel and Ethernet. The standalone solution provides a significant amount of memory available in the equipment that can be useful for creating user defined patterns. The integrated BERT solution utilizes internal BlockRAM within the FPGA for some of the simple patterns and relies upon memory available in the lab machine that runs the software for more advanced and user defined patterns.

Rates

The high-speed serial transceivers within FPGA have the capability to operate from 622Mbps up to 11.1Gbps requiring the BERT solution to support this range as well. To accommodate this, the BERT solution must be capable of providing different reference clocks and settings to match the given line rate, as well as multiple clock references. Using the FPGAs available clocking resources, this is no problem for the integrated BERT core. It is not a problem for the stand alone BERT solution ether as long as it can support multiple high-speed serial line rates. It is important to notes that as the line rate increases so does the cost of the stand alone BERT solution.

Number of transceivers

FPGA devices support a variable number of high-speed serial transceivers within a single device. While it is easy to scale the integrated BERT solution to test simultaneously any of the available high-speed serial transceivers within the FPGA, the stand alone BERT solution can support only a single transceiver at a time.

User interface

The standalone BERT solution is a self contained unit that incorporates the pattern generator, error detection and graphical user interface that puts control in one convenient location. Assuming the user has some way to configure the FPGA; this is the minimal set of equipment needed to perform BER analysis. The integrated BERT solution requires a lab machine, typically a PC, to control the BERT core placed within the FPGA. Utilizing the JTAG interface, this lab machine can be used to configure the FPGA as well as communicate with the integrated BERT core. The pattern generator and error detector are both controlled from a graphical user interface that is run on the lab machine. Both the stand alone and integrated BERT solution will automatically detect which high-speed serial transceivers are operating and both enable control of the transceiver clock settings.

Advanced features

Both the standalone and integrated BERT solutions enable advanced features that can simplify the evaluation phase of high speed serial technology within an FPGA. The ability to inject jitter or a single error bit in the transmitted data are available in both the stand alone and integrated BERT solutions. One very important capability that is unique to the integrated BERT solution is the ability to automatically setup of the individual high speed serial settings. At high-speeds greater than 3.125Gbps, techniques such as receiver equalization and transmit pre-emphasis are used to compensate for frequency dependant signal attenuation and board and trace variation, respectively. Determining the ideal settings for a given high speed serial channel can be time consuming and tedious. Because the integrated BERT solution has direct access to the settings with the FPGA transceiver technology, it can be set up to automatically vary these settings and take corresponding BER measurements. Based upon these results, a margin curve, similar to a bathtub curve, can be displayed showing the optimal settings and predicated BER.

Cost

Stand alone BERT equipment ranges in price from just over $10,000 to well over $100,000. For the BERT features described in this article, one can expect to pay around $85,000. Given the price of FPGA on-chip debug and verification solutions available today, an integrated BERT solution would likely cost less than $500. Even if one were to add the cost of a dedicated lab computer to run the integrated BERT software, there is still a compelling cost advantage to the integrated BERT solution.

This comparison covers only one phase of a high-speed serial design, covering restricted test conditions for a single FPGA without a working application. Within these conditions, the integrated BERT solution compares favorably with a stand alone BERT solution. Table 1 summarizes and compares both solutions in terms of setup, supported capabilities, user interface, advanced features, and cost. As users address the next phases of high speed serial design - design validation and margin analysis, and system operation - the stand alone BERT solution will provide advanced levels of support and the necessary functionality to drive a design to completion. To be sure, the integrated BERT solution will have its place as well. Ideally, the optimal approach would combine the best of both worlds in a single, well-integrated solution.

Table 1. Comparison of standalone BERT and integrated BERT.

Conclusion

The flexibility, access, and control inherent in the FPGA architecture enable on-chip debug and verification. These same techniques can be applied to high speed serial technology to provide a level of test and verification access previously available only with external stand alone BERT equipment. The technology exists today to integrate a BERT solution within the FPGA that provides the same functionality needed during the technology evaluation phase of a high speed serial design. The true potential of the integrated solution is the unique access to silicon features currently not available from outside the FPGA. The ability to automatically determine optimal high-speed serial transceiver settings will significantly reduce the time and effort necessary to set up and verify high speed serial technology within the FPGA. Not only is an integrated BERT solution a reality, it is a very cost effective solution that engineers working with high speed serial technology cannot afford to ignore.

By Brent Przybus