Technical Details

The Effective Model Selection for DSDs Algorithm

This section provides a summary of the algorithm used in the Fit Definitive Screening platform. For further details, see Jones and Nachtsheim (2016).

Decomposition of Response

The Effective Model Selection algorithm expresses the response, Y, in terms of two responses YME and Y2nd, so that Y = YME + Y2nd.

–

YME is the predicted value obtained from a regression of Y on the main effects and fake factors.

There is no need to include the block factor in YME because of the fold-over structure of the design. The block factor is included in Y2nd.

–

Y2nd is given by Y2nd = Y - YME.

Note: In a DSD, the columns YME and Y2nd are orthogonal.

The analysis proceeds in two stages:

•

Stage 1: The response Y is used to identify main effects. Stage 1 identifies the main effects that are considered active.

•

Stage 2: The response Y2nd is used to identify second-order effects. Stage 2 considers all second-order terms in the active main effects from Stage 1 and determines a subset of these containing effects considered to be active.

Note: If there is a blocking factor, it is included in the Stage 2 list of effects even if it is not significant.

Stage 1 Methodology

The Stage 1 methodology depends on whether the design contains fake factors or centerpoint replicates.

Case 1: Fake Factors or Centerpoint Replicates Available

Using the fake factors or center point replicates, an estimator of error variance that is independent of the model is constructed. Assuming that there are no active third or higher odd order effects, this estimate is unbiased.

Using YME, main effects are tested against this estimate. Main effects with p-values less than a threshold p-value are considered active. The threshold values are the following:

–

For one error degree of freedom, the threshold value is 0.20.

–

For two error degrees of freedom, the threshold value is 0.10.

–

For more than two error degrees of freedom, the threshold value is 0.05.

If no main effect has a p-value less than the threshold value, conclude that there are no active main effects and no active two-factor effects. The procedure terminates.

If active main effects are found, then variability from the inactive main effects is pooled into the error variance constructed in (1).

Case 2: No Fake Factors or Centerpoint Replicates Available

In this case, there is no model-independent estimator of error variance available. Subsets of main effects are tested sequentially against an estimate of error variance constructed from the inactive main effects. Suppose that there are m main effects.

The absolute values of the estimated effects, using YME as the response, are ordered from largest to smallest.

For each 1 ≤ i < m, the effect with the ith largest absolute value is tested against the adjusted residual sum of squares for the model containing that effect and all effects with larger absolute values.

The effects in the model with the smallest p-value are considered to be the active effects.

If active main effects are found, then variability from the inactive main effects is used to construct an estimate of error variance, using YME as the response.

Note: For the Fit Definitive Screening procedure to work properly in Case 2, at least one of the main effects must be active and at least one must be inactive. If no main effects are active, or if all main effects are active, the procedure will identify a set of main effects, but the procedure for arriving at that subset is compromised.

Stage 2 Methodology

In Stage 2, only second-order effects involving the factors whose main effects are active are considered. Stage 2 uses a guided subset selection procedure. The goal is to continue to add second-order effects to the model as long as the RMSE from Stage 2 exceeds the RMSE from Stage 1. When the Stage 2 RMSE is less than or equal to the Stage 1 RMSE, this indicates that there are no additional second-order effects to add to the model.

The same threshold values are used as in Stage 1:

•

For one error degree of freedom, the threshold value is 0.20.

•

For two error degrees of freedom, the threshold value is 0.10.

•

For more than two error degrees of freedom, the threshold value is 0.05.

The variability for Y2nd is tested against the error estimate from Stage 1 to determine if there is additional variability due to second-order effects.

–

If the p-value for this test exceeds the threshold value the procedure terminates and no active second-order effects are identified.

If the p-value for this test is less than or equal to the threshold value, then subsets of size k, k = 1,2,3,... are successively tested, starting with k = 1.

For each k, the residual sum of squares for each subset of that size is tested against the error estimate from Stage 1. The subset with the smallest RMSE is identified.

The procedure continues until a k is found whose RMSE is smaller than the Stage 1 RMSE.

The effects in the subset preceding the one that corresponds to the terminal value of k are considered to be the active two-factor effects.