Power Analysis

In the Compare Designs report, the Power Analysis section provides the power of tests for the effects specified in the Model section. Power is the probability of detecting an active effect of a given size. The Power Analysis section helps you evaluate and compare the ability of your designs to detect effects of practical importance.

The Power Analysis section provides the power of tests for individual model parameters and for whole effects. It also provides a Power Plot and a Power versus Sample Size plot.

Power depends on the number of runs, the significance level, and the estimated error variation. For more information about how power is calculated, see “Power Calculations”.

Figure 17.11 Power Analysis for Three Designs 

Power Analysis Report

Specify values for the Significance Level and Anticipated RMSE to calculate the power of the tests for the model parameters. Enter coefficient values that reflect differences that you want to detect as Anticipated Coefficients. To update the results for all designs, click Apply Changes to Anticipated Coefficients.

Significance Level

The probability of rejecting the hypothesis of no effect, if it is true. The power calculations update immediately when you enter a value.

Anticipated RMSE

An estimate of the square root of the error variation. The power calculations update immediately when you enter a value.

Tests for Individual Parameters

The Power Analysis table contains a list of model terms, the specified anticipated coefficient for that term, and the power for each design.

Term

The model term associated with the coefficient being tested.

Anticipated Coefficient

A value for the coefficient associated with the model term. This value is used in the power calculations. When you update a value in the Anticipated Coefficient column, click Apply Changes to Anticipated Coefficients to update the calculations.

Note: The anticipated coefficients have default values of 1 for continuous effects. They have alternating values of 1 and –1 for categorical effects.

Power

The probability of rejecting the null hypothesis of no effect when the true coefficient value is given by the specified Anticipated Coefficient. Calculations use the specified Significance Level and Anticipated RMSE. For more information about the power calculation, see “Power for a Single Parameter”.

– For a coefficient associated with a numeric factor, the change in the mean response from the high to low setting (based on the model) is twice the coefficient value.

– For a coefficient associated with a categorical factor, the change in the mean response (based on the model) across the levels of the factor equals twice the absolute value of the anticipated coefficient. For more information about power for categorical factors, see Tests for Categorical Effects with More Than Two Levels

– The power values are colored according to a color gradient that appears under the Apply Changes to Anticipated Coefficients button. You can control the color legend using the options in the Power Analysis red triangle menu. See Color Legend.

Note: For supersaturated designs, where the number of parameters to be estimated exceeds the number of runs, the Power Analysis section lists only the set of effects that can be estimated.

Apply Changes to Anticipated Coefficients

When you set a new value in the Anticipated Coefficient column, click Apply Changes to Anticipated Coefficients to update the Power values.

Tests for Categorical Effects with More Than Two Levels

If your model contains a categorical effect with more than two levels, then the following columns appear below the Apply Changes to Anticipated Coefficients button:

Effect

The categorical effect.

Power

The power calculation for a test of no effect. The null hypothesis for the test is that all model parameters corresponding to the effect are zero. The difference to be detected is defined by the values in the Anticipated Coefficient column that correspond to the model terms for the effect. The power calculation reflects the differences in response means determined by the anticipated coefficients.

Calculations use the specified Significance Level and Anticipated RMSE. For more information about the power calculation, see “Power for a Categorical Effect”.

Power Plot

The Power Plot shows the power values from the Power Analysis in graphical form. The plot shows the power for each effect and for each design in a side-by-side bar chart.

Figure 17.12 Power Plot for Three Designs 

Power versus Sample Size

The Power versus Sample Size plots appears only when the designs that you are comparing differ in run size. The plots enable you to see how sample size affects power for each effect in the model. It conveys the same information as is in the Power Plots graph, but in a different format. The power values at integer sample sizes are connected with line segments.

Figure 17.13 Power versus Sample Size Profiler for Three Designs