Supersaturated Screening Designs

It is common for brainstorming sessions to identify dozens of potentially active factors. Rather than reduce the list without the benefit of data, you can use a supersaturated design.

In a saturated design, the number of runs equals the number of model terms. In a supersaturated design, the number of model terms exceeds the number of runs (Lin, 1993). A supersaturated design can examine dozens of factors using fewer than half as many runs as factors. This makes it an attractive choice for factor screening when there are many factors and experimental runs are expensive.

Limitations of Supersaturated Designs

There are drawbacks to supersaturated designs:

•

If the number of active factors is more than half the number of runs in the experiment, then it is likely that these factors will be impossible to identify. A general rule is that the number of runs should be at least four times larger than the number of active factors. In other words, if you expect that there might be as many as five active factors, you should plan on at least 20 runs.

•

Analysis of supersaturated designs cannot yet be reduced to an automatic procedure. However, using forward stepwise regression is reasonable. In addition, the Screening platform (Analyze > Specialized Modeling > Specialized DOE Models > Fit Two Level Screening) offers a streamlined analysis.

Generate a Supersaturated Design

In this example, you want to construct a supersaturated design to study 12 factors in 8 runs. To create a supersaturated design, you set the Estimability of all model terms (except the intercept) to If Possible.

Note: This example is for illustration only. You should have at least 14 runs in any supersaturated design. If there are as many as four active factors, it is very difficult to interpret the results of an 8-run design. See Limitations of Supersaturated Designs.

Select DOE > Custom Design.

Type 12 next to Add N Factors.

Click Add Factor > Continuous.

Click Continue.

In the Model outline, select all terms except the Intercept.

Click Necessary next to any effect and change it to If Possible.

Setting the effects to If Possible ensures that JMP uses the Bayesian D-optimality criterion to obtain the design.

Figure 4.12 Factors, Model, and Number of Runs

In the Alias Terms outline, select all effects and click Remove Term.

This ensures that only the main effects appear in the Color Map on Correlations. This plot is constructed once the design is created.

Select Simulate Responses from the red triangle menu.

This option generates random responses that appear in your design table. You will use these responses to see how to analyze experimental data.

Keep the Number of Runs set to the Default of 8.

Note: Setting the Random Seed in step 9 and Number of Starts in step 10 reproduces the design shown in this example. In constructing a design on your own, these steps are not necessary.

(Optional) From the Custom Design red triangle menu, select Set Random Seed, type 1008705125, and click OK.

10.

(Optional) From the Custom Design red triangle menu, select Number of Starts, type 100, and click OK.

11.

Click Make Design.

12.

Click Make Table.

Do not close your Custom Design window. You return to it later in this example.

The design table (Figure 4.13) and the Simulate Responses window (Figure 4.14) appear.

Figure 4.13 Design Table with Simulated Responses

The response columns, Y and Y Simulated, are initiated with the same simulated values. The values are random values from a N(0, s) distribution. where s is the RMSE from the power analysis dialog with a default of 1. The Y Simulated values are updated with randomly generated values using the model defined by the parameter values in the Simulate Responses window. The Y column is intended for your true responses after the experiment has be run.

Figure 4.14 Simulate Responses Window

The Simulate Responses window shows coefficients of 0 for all terms, a Normal distribution selection and Error σ of 1. The values in the Y and Y Simulated column currently reflect only random variation. Notice that the model coefficients are set to 0 because not all coefficients are estimable.

13.

Change the values of the coefficients in the Simulate Responses window as shown in Figure 4.15.

Figure 4.15 Parameter Values for Simulated Responses

14.

Click Apply.

The response values in the Y Simulated column change. See Figure 4.16.

Note: The response values are randomly generated. Your values will not match those in Figure 4.16 exactly.

Figure 4.16 Y Simulated Response Column with X1 and X11 Active

In your simulation, you specified X1 and X11 as active factors with large effects relative to the error variation. For this reason, your analysis of the data should identify these two factors as active.

Analyze a Supersaturated Design Using the Screening Platform

The Screening platform provides a way to identify active factors. The design table in Figure 4.16 contains three scripts. The Screening script analyzes your data using the Screening platform (located under the Analyze > Specialized Modeling > Specialized DOE Models > Fit Two Level Screening menu).

In the Table panel of the design table, click the green triangle next to the Screening script.

Figure 4.17 Screening Report for Supersaturated Design

The factors X1 and X11 have large contrast and Lenth t-Ratio values. Also, their Simultaneous p-Values are small. In the Half Normal Plot, both X1 and X11 fall far from the line. The Contrasts and the Half Normal Plot reports indicate that X1 and X11 are active. Although X12 has an Individual p-Value less than 0.05, its effect is much smaller than that of X1 and X11.

Because the design is supersaturated, p-values might be smaller than they would be in a model where all effects are estimable. This is because effect estimates are biased by other potentially active main effects. In Figure 4.17, a note directly above the Make Model button warns you of this possibility.

You might also want to check whether the effects that appear active could be highly correlated with other effects. When this occurs, one effect can mask the true significance of another effect. The Color Map in Figure 4.19 displays absolute correlations between effects.

Click Make Model.

The constructed model contains only the effects X1, X11, and X12.

Click Run in the Model Specification window.

Figure 4.18 Parameter Estimates for Model

Note that the parameter estimates for X11 and X1 are close to the theoretical values that you used to simulate the model. See Figure 4.15, where you specified a model with X1 = 10 and X11 = 10. The significance of the factor X12 is an example of a false positive.

In your Custom Design window, open the Design Evaluation > Color Map on Correlations outline.

Figure 4.19 Color Map on Correlations Outline

With your cursor, place your mouse pointer over cells to see the absolute correlations. Notice that X1 has correlations as high as 0.5 with other main effects (X4, X5, X7). (Figure 4.19 uses JMP default colors.)

Analyze a Supersaturated Design Using Stepwise Regression

Stepwise regression is another way to identify active factors. The design table in Figure 4.16 contains three scripts. The Model script analyzes your data using stepwise regression in the Fit Model platform.

In the Table panel of the design table, click the green triangle next to the Model script.

Change the Personality from Standard Least Squares to Stepwise.

Click Run.

In the Stepwise Fit for Y report, change the Stopping Rule to Minimum AICc.

For designed experiments, BIC is typically a more lenient stopping rule than AICc as it tends to allow inactive effects into the model.

Click Go.

Figure 4.20 Stepwise Regression for Supersaturated Design

Figure 4.20 shows that the selected model consists of the two active factors, X1 and X11. The step history appears in the bottom part of the report. Keep in mind that correlations between X1 and X11 and other factors could mask the effects of other active factors. See Figure 4.19.

Note: This example defines two large main effects and sets the rest to zero. Real-world situations can be less likely to have such clearly differentiated effects.