May 1, 2005 -- The purpose of integrated circuit test is to help segregate good devices from those that are defective to improve manufacturing yield -- reducing defective parts per million to as low as 100 defective parts per million and thereby increasing shipped yield.

These low-defect levels are achieved through automatic test pattern generation (ATPG) algorithms that target faults that are abstractions of defects. By making some design modifications to assist ATPG, it is possible to automate the task of targeting defect-based faults and achieve high fault coverage/efficiency with a reasonable number of test patterns. Like any other automation activities completed in ATPG, the results of this process can then be scrutinized in benchmarks, and tool improvements can be monitored over successive generations of the product to achieve more efficient IC test automation.

ATPG performance is characterized by fault efficiency, run time and pattern count. The fault efficiency represents the percentage of faults that ATPG has been able to resolve successfully. The run time represents the amount of CPU time it takes to generate the results. The pattern count represents the number of tests generated to achieve the fault coverage. While higher fault coverage is desirable -- along with lower run times and pattern counts -- the three factors cause the algorithm to evaluate a full complement of tradeoffs and produce more realistic results. For example, lower CPU times and pattern counts are achievable at the expense of fault coverage. Similarly, lower pattern-counts are achievable through complex compaction schemes -… but at the expense of longer run times.

For this reason, it is very difficult to achieve realistic and accurate ATPG performance analysis when running disparate tools - or different versions of disparate tools - to determine fault efficiency, run time and pattern count. Comparison is possible when two of the three results have the same value . This is a very low probability event which is difficult to achieve by tweeking the interface to the tools.

In this article, we evaluate how the ATPG fault efficiency, runtime and pattern count might be matched during multiple executions to achieve a comparison between different ATPG executions. A straightforward method is to compare the runtime number against the fault coverage number when the two numbers are within a certain percentage of each other … provided the pattern count number is also within a reasonable percentage of the other two. However, this comparison, alone, will not produce a result for most comparative ATPG executions. For that, this article goes a step further to show how we can achieve equivalent numbers for all executions by computing a metric that equalizes the fault efficiency and pattern count differences in the numbers.

Relationship between ATPG numbers

The following figure shows the relationship between ATPG runtime, fault efficiency, and pattern count for a sample run.

Figure 1. Relationship between ATPG runtime, fault efficiency, and pattern count for a sample run.

It can be seen that outside of the boundaries the two curves are quite linear. After an initial ramp up in fault coverage the faults detected over the run is practically a straight line. The last 10 percent of coverage is achieved in 90 percent of the overall run time. The number of patterns generated grows quite linearly during the ATPG run with some anomalies around the ends of the curve. For most purposes the run time is and the pattern count have a linear relationship. While the relationships are similar across designs the rate at which the slopes of the curves vary with the design.

Comparing two ATPG runs for CPU time

Let us assume that the two versions of the same tool are being executed. The older version run provides results of run time t, pattern count p, and fault efficiency f. The newer version of the same tool with the same settings ends with a run time T, pattern count P and fault efficiency of F.

If the pattern count and fault coverage are the same for the two executions we are interested in the Improvement Factor (IF) which is the ratio of the older run time to the newer run time (assuming the newer version of the tool is better).

However, if there is a difference in fault coverage the associated time to achieve that fault coverage is to be accounted appropriately. Using the linear relationship between faults and run time where 90% of the run time is used to achieve the last 10% of the fault coverage, the difference in the fault accounting is converted to time and its credit is to be given accordingly. As a result:

The difference in the pattern count can similarly be translated to time, (p - P)xt/P . However, while equalizing the faults the time taken to detect those extra faults was already accounted for. By simply taking credit for the difference in patterns the patterns generated to detect the difference in faults accounted for is double counted. The patterns used to detect the extra faults is computed as
(f - F)x0.9p/0.1f and removed from the pattern accounting. Hence the improvement factor is given by:

Computing the improvement factor

To show the calibration of this equation, we ran the same tool under different conditions to produce different numbers. With compaction turned on, 422 patterns were generated to achieve 97.11% fault efficiency in 146.81 cpu sec. With compaction turned to a lower setting, the same tool produced 444 patterns to achieve 96.96% fault efficiency in 134 cpu sec. Thus, the run time of the execution is 1.1 times faster. When the IF equation the equivalent improvement factor is computed to be 1.0, which is true as the code base was the same and all we did was change the parameters.

The results were used to compare the performance improvements between two versions on the same product. The following table shows the IF numbers for the data collected.

As the numbers show, simple run time comparisons are unfair. When one takes into account the changes in fault efficiency and pattern count the IF reflects the overall run time comparison. Design A showed a 1.83 times increase in performance and Design B showed a 2.05 times increase in performance between the two versions of the same tool.

By Rohit Kapur

and

Thomas W. Williams, Ph.D.

Kapur is a Synopsys Scientist working in the area of Test Automation. Rohit is an IEEE Fellow and chairs the standards activities in the area of testing.

Williams is a Synopsys Fellow working in the area of test automation. He is an IEEE Fellow and has received numerous best paper awards from the IEEE and ACM, is the founder or co-founder of a number of workshops and conferences dealing with testing, and was twice a Distinguished Visitor lecturer for the IEEE Computer Society.