Example of a Custom Design Analysis

Use a regression model to evaluate the results from a custom designed experiment.

1. Select Help > Sample Data Folder and open Design Experiment/Wine Data.jmp.

2. In the Table panel, click the green triangle next to the Model script.

Figure 4.10 Model Specification Window for the Wine Experiment 

Notice that Rater, the blocking factor, is added as a fixed effect. This is appropriate because the five raters were specifically chosen and are not a random sample from a larger population. If the raters were a random sample from a larger population, the blocking factor would be random effect.

3. Click Run.

Interpret the Full Model Results

The model output from the custom design example is shown below.

Figure 4.11 Partial Model Fit Results 

Note the following:

• The Actual by Predicted Plot shows no obvious evidence of lack of fit.

• The model is significant, as indicated by the Actual by Predicted Plot and by the p-value beneath it.

• The Effect Tests report indicates that seven of the model terms are significant at the 0.05 level. Field, Temperature, and Barrel Age are not significant.

• The Effect Summary report lists these effects in decreasing order of significance. Larger logworth values correspond to smaller p-values and greater significance.

Reduce the Model

Reduce the model for the custom design results by removing the effects that you identified as inactive:

1. In the Effect Summary report, press Control and select Temperature, Field, and Barrel Age.

2. Click Remove.

The report updates to show the model fit with these three effects removed.

Interpret the Model Results with the Profiler

The Actual by Predicted Plot for the reduced model shows no lack of fit issues. The Effect Summary and the Effect Test report show that the remaining seven terms are significant at the 0.05 level. Use the prediction profiler to further explore your reduced model.

Figure 4.12 shows the Prediction Profiler. The response has a specified goal of Maximize, with lower and upper limits of 0 and 20. These limits were set during the design creation and thus, a Response Limits column property was saved to the Rating column in the Custom Design table. The Prediction Profiler uses the Response Limits information to construct a Desirability function, which appears in the right-most plot in the top row in Figure 4.12. The bottom row displays Desirability traces.

The first six plots in the top row show traces of the predicted model. For each factor, the line in the plot shows how Rating varies when all other factors are set at the values defined by the red dashed vertical lines. By default, the profiler appears with categorical factors set at their low settings. By varying the settings for the factors, you can see how the predicted Rating for wines changes. Notice that a confidence interval is given for the mean predicted Rating.

Observe that Rater is not included among the factors shown in the profiler. This is because Rater is a block variable. You included Rater to explain variation, but Rater is not of direct interest in terms of optimizing process factor settings. The predicted Rating for a wine with the given settings is the average of the predicted ratings for that wine by all raters.

Figure 4.12 Profiler for Reduced Model 

Optimize Factor Settings

Use the prediction profiler to identify optimal settings based on your custom design results. You would like to identify settings that maximize Rating across raters.

1. Click the Prediction Profiler red triangle and select Optimization and Desirability > Maximize Desirability.

The red dashed vertical lines in the Prediction Profiler update to show optimal settings for each factor. The optimal settings result in a predicted rating of 19.925. In general, there can be different sets of factor settings that result in the same optimal value.

Figure 4.13 Prediction Profiler with Factor Settings Optimized 

2. To see predicted ratings for all runs, save the Prediction Formula. Click the Response Rating red triangle and select Save Columns > Prediction Formula.

A column called Pred Formula Rating is added to the data table. Note that one of the runs, row 33, was given the maximum rating of 20 by Rater 5. The predicted rating for that run by Rater 5 is 19.550. But the row 33 trial was run at the optimal settings. The predicted value of 19.925 given for these settings in the Prediction Profiler is obtained by averaging the predicted ratings for that run over all five raters.

Lock a Factor Level

You can lock factors in the profiler to explore optimization under specific conditions. When you maximized the design response, you learned that the optimal rating is achieved with the Dijon variety of grapes (Figure 4.13). It would be cost-prohibitive to replant the fields that are growing Bernard grapes with young Dijon vines. Therefore, you need to find optimal process settings and the predicted rating for Bernard grapes.

1. In the Variety plot of the Prediction Profiler, drag the red dashed vertical line to Bernard.

2. Press Control and click in one of the Variety plots.

The Factor Settings window appears.

3. Select Lock Factor Setting and click OK.

4. Click the Prediction Profiler red triangle and select Optimization and Desirability > Maximize Desirability.

Figure 4.14 Prediction Profiler with Optimal Settings for Bernard Variety 

The optimal settings are unchanged because the model contains no interaction terms. The predicted rating at these settings is 17.975.

Add the Rater to the Profiler

If you want to see the Profiler traces for the levels of Rater, perform the following steps:

1. Click the Prediction Profiler red triangle and select Reset Factor Grid.

A Factor Settings window appears with columns for all of the factors, including Rater. The box under Rater and next to Show is not checked. This indicates that Rater is not shown in the Prediction Profiler.

2. Check the box under Rater in the row corresponding to Show.

3. Deselect the box under Rater in the row corresponding to Lock Factor Setting.

4. Click OK.

The Profiler updates to show a plot for Rater.

5. Click in either plot above Rater.

Figure 4.15 Profiler for Reduced Model Showing Rater 

A dashed vertical red line appears. Drag this line to see the traces for each of the raters. Keep in mind that Variety is still locked at Bernard. To unlock Variety, press Control and click in one of the Variety plots. In the Factor Settings window that appears, deselect Lock Factor Setting.