The estimate of the standard deviation, s, and the associated degrees of freedom for the t-test are obtainable from our measurements. Since the two sets of measurements were run on different days, we should be concerned that one set of measurements could be offset relative to the other set. Let us therefore calculate the s value by a method that is not vulnerable to an offset between the two sets of measurements.

We first note that an offset between the sets of measurements will not influence the values of the calculated effects. Let us therefore consider the differences between the effects as calculated for the above example (see table 4, column 7). Since we are considering the same effects from the two sets of experiments, the statistically expected values of the differences between the effects are zero. The variance of the difference is therefore the expected value of the squared differences.

An estimate of the expected value of (d2) is obtained by simply averaging the squares of the differences listed in table 4, column 7. Our calculated estimate is 384/7=54.9.

We next note that the variance of the difference (between the duplicated effects) is the sum of the variances of the two effects. The variances of the two effects should be the same since the two sets of experiments were done in the same laboratory. Equation 2b described the sample estimate for the square root of the variance of an effect. Therefore:

This quantity, in absolute value, it is slightly less than the 5% critical t-value of 2.36. It is not quite statistically significant. The C factor describes the effect of a small dilution, as one might get from not properly wiping dry the glass electrode.

As mentioned above, if the effect of any factor is too large one may wish to tighten the specification for that factor. The goal, of course, is to reduce the interlaboratory variability. More detailed discussions of the pH measurement experiments are presented in Part II.

copying the first row after shifting it one place to the right and putting the last sign of row 1 in the first position of row 2. This type of cyclic shifting should be done a total of N-2 times, after which a final row of all minus signs is added. The result of this procedure for the N=8 Plackett-Burman design is given in table 1.

Some ruggedness test studies may not involve exactly N-1 factors. If we believe, for example, that only five instead of seven factors might influence the measured results, we might use two dummy factors. For one of the dummy factors we might pour a solution with our left hand for the (+) level and with our right hand for the (-) level. The calculated "effect" for the dummy factor should be small and should simply reflect our random errors of measurement.

Conclusions

A straightforward explanation of the statistical technique of ruggedness testing has been presented. Orthogonal Plackett-Burman designs allow the ruggedness test user to efficiently evaluate the effects of the separated variables on a measurement process. The present article (Part I) deals with the common situation where twofactor and higher order interactions can be safely ignored.

References

[1] Plackett, R. L., and J. P. Burman, The Design of Optimum Multifactorial Experiments, Biometrika, Vol. 33, 305–325 (1946). [2] Yates, F., Complex Experiments, J. Roy. Statistical Soc. (Supplement), Vol. 2, 181-247 (1935).

[3] Youden, W. J., Designs for Multifactor Experimentation, Industrial and Engineering Chemistry, Vol. 51, 79A-80A (1959). [4] Diamond, W. J., Practical Experimental Designs for Engineers and Scientists, pp. 103 and 110, Lifetime Learning Publications, Belmont, CA (1981).

[5] Marinenko, George; Robert C. Paule, William F. Koch, and Melissa Knoerdel, Effect of Variables on pH Measurement in Acid-Rain-Like Solutions as Determined by Ruggedness Tests, J. Res. Natl. Bur. Stand. 91-1 (1986).

Introduction

This paper is a continuation of the preceding (Part I) article which introduced the general principles of ruggedness testing. To be read in conjunction with Part I, it describes the effects of interactions on a measurement process and presents procedures for separating main effects and two-factor interactions.

Interactions and That Confounded Confounding

From Part I we know that an N measurement experiment can be used to determine N-1 main factors, provided the interactions are small. It is usually the case, in experiments involving well-behaved functions of the measurement variables, that when the main effects are small the associated interactions are very small. The

About the Authors: Robert C. Paule is a physical scientist assigned to the NBS National Measurement Laboratory (NML). George Marinenko and William F. Koch are chemists in NML's Inorganic Analytical Research Division in which Melissa Knoerdel, a student, serves the Division during summer vacations.

interactions are, in effect, the non-ideal departures from a simple additive model consisting of only constant main effects. Nevertheless, situations occasionally arise in which interactions are important.

In an eight-run, seven-factor experiment each main effect is confounded with 15 different possible interactions. Of the 15 interactions, the number and types are as follows: 3 two-factor, 4 three-factor, 4 four-factor, 3 five-factor, and 1 six-factor. Table 5,' which corresponds to the Yates-Youden design (see table 2 of part I), shows each of the main effects and the associated twoand three-factor interactions.

The Yates-Youden design (and the Plackett-Burman designs of a size such that N=2, where k is a positive integer) allow a relatively easy separation and determination of the more important confounding interactions. These designs allow one to use the Multiplication Rule for signs. The Multiplication Rule [4]1 states that the pairwise multiplication of like signs produces a (+) and that of unlike signs produces a (-). Thus, looking at table 2 of Part I, the row pairwise multiplication of the signs for columns B and D produces the following column for the BD interaction:

BD

++

(see table 3 of Part I), and a third set of pH measurements which was made with all levels of the design reversed. Let us now consider the combined results from the first and third sets of measurements.

An examination of the signs of table 6, and the use of the Multiplication Rule, will show that the two-factor interactions - BD, -CE, and -FG (which were grouped together in column 1 of table 5) still have an identical sign pattern but that this pattern is now different from the A main effect. The BD interaction has the following sign pattern (−−−−++++ ++++). One can see that the last half of the interaction sign pattern is a repetition of the first half whenever an even-number of factors is multiplied together, but that the last half has a sign reversal whenever an odd-number of factors is multiplied together.

Note that this column is the exact opposite of the signs of column A, given in table 2. Thus, -BD is the same as A. It is inseparable from A in the eight-run, seven-factor experiment since the values of the eight measurements are combined in an identical manner. Similar multiplications of signs shows that A-CE-FG. Multiplication of signs of the rows of columns "BC" and G produces the three-factor interaction BCG which is observed to be the same as factor A. Column "BC" can be simply obtained by using minus column F (see table 5). The confounding of all higher order interactions can be obtained by an extension of this general procedure.

If we wish to protect ourselves from misinterpretations due to large interactions, we must make more than N measurements for determining the N-1 main factors. To evaluate the main effects and all interactions, we must do the full factorial experiment. For seven factors this requires 128 measurements. Usually, however, one does not have to go this far. A reasonable compromise experiment consists of making two sets of N measurements which allow the separation of each of the main effects from the two-factor interactions. This compromise, however, does not separate among each of the two-factor interactions, and in addition it assumes that three-factor and other higher order interactions are unimportant. If we demand more information, then we have no choice but to make more measurements!

Let us again consider a seven factor pH experiment involving 2N (= 16) measurements. This time we will use the previously reported first set of pH measurements

From table 6, and the Multiplication Rule, one can see that the three two-factor interactions within each column of table 5 are not separable from one another, but that they are separable from the main effects. The threefactor interactions are not separable from the main effects. A further consideration of table 6 will show that the main effects and their odd-numbered interactions are not separable from one another, but that they are separable from all of the even-numbered interactions. The nearest higher order interaction contamination for either the odd- or the even-numbered interactions is now two-factor multiples distant. If the magnitude of the interactions decreases as one goes toward higher order

interactions, then one has achieved a practical separation (isolation) of the main effects and of the groups of two-factor interactions.

Main effects A-G can be calculated from the data of table 6 by use of eq (1) (from Part I). The calculated respective effects are +51, −2, +4, +6, +27, +79, and -0.4 milli-pH units. The two-factor interactions are calculated in the same manner as the main effects. Note that the value of the "new N" in eq (1) is the combined N from both sets of measurements (new N=16). As shown above, the sign pattern for the BD interaction is (----++++----++++). The value for the combined (BD, CE, FG) interactions is +11 millipH units. The other two-factor interactions can be calculated in a similar manner. Finally, we note from table 6 that if an offset had occurred between the first and third set of measurements, it would not affect the calculations of the main effects or the interactions. This immunity to offsets between the different sets of measurements is a consequence of using the Plackett-Burman based design. The PB-design will always have an equal number of positive and negative signs within each set so that the absolute level of the sets of measurements will not affect the calculations.

For the set of eight measurements, one keys the measurements into memory registers 1-8, respectively, and then calculates the last term of eq (9a) which is two times the average of the eight measurements. This quantity is stored in memory register 9. In order to minimize the chance of error, it is advisable to use the measurement results that are stored in memory registers 1-8 to calculate this latter quantity. One then simply uses eq (9a) and the columns of table 7 to calculate the various effects:

5 6

3

4

2

3

4 4

7 7 8 8 8

5

6 6

2 2

5

8

Observed pH (milli-pH units)

2904

3015

3006

2964

2999

3055

3049

2949

Average 2992.625

The reverse sign PB-design listed in the bottom of table 6 can be similarly rewritten and used with eq (9a) to again calculate effects A-G.

Table 8 lists the calculated effects from the three sets of eight pH measurements that have been previously reported in tables 3 and 6. The actual, chronological order used for making our measurement sets consisted of the table 2 design of Part I, the reverse-sign design, the repeat table 2 design, and occasionally a repeat of the reverse-sign design. The labeling from our pre