Model Selection Method

Processes | Predictive Modeling | Model Selection Method

Model Selection Method

Use this drop-down menu to select the method to be used for selecting the final predictor variables.

Available methods are detailed in the following table:

Model Selection Method

Definition

None

Choose this option to use all of the variables and backward starts with all of them

Caution : Choosing this option can result in long run times when there are thousands of variables.

Forward

Choose this option to perform a forward search, beginning with no variables in the model and adding the most significant ones one at a time.

Backward

Choose this option to perform a backward search. This option begins with using all of the variables and then removes the least significant ones one at a time.

Caution : Choosing this option can result in long run times when there are thousands of variables.

Stepwise

Choose this option to perform a forward stepwise method.

LAR

Choose this option to perform a least-angle regression ¹ , which begins with no effects. The parameter estimates at any step are shrunken when compared to the corresponding least squares estimates. If the model contains classification variables, then these classification variables are split. ² See the SPLIT option in the CLASS statement of SAS PROC GLMSELECT for details.

LASSO

Choose this option to add and delete parameters based on a version of ordinary least squares where the sum of the absolute regression coefficients is constrained ³ . If the model contains classification variables, then these classification variables are split. 2

Elastic Net

Choose this option to augment the data and use a LASSO fit in accordance with an Elastic Net. ⁴

1

Efron, B., T. Hastie, et al. (2004). Least angle regression. Annals of Statistics 32(2): 407–499.

2

See the SPLIT option in the CLASS statement of the SAS PROC GLMSELECT documentation for details.

3

Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. R. Statist. Soc. B 58: 267–288.

4

See http://www.sas-programming.com/2011/04/elasticnet-in-sas.html for details.

To Specify the Method:

Select the method using the drop-down menu.

Model Selection Method	Definition
None	Choose this option to use all of the variables and backward starts with all of them Caution : Choosing this option can result in long run times when there are thousands of variables.
Forward	Choose this option to perform a forward search, beginning with no variables in the model and adding the most significant ones one at a time.
Backward	Choose this option to perform a backward search. This option begins with using all of the variables and then removes the least significant ones one at a time. Caution : Choosing this option can result in long run times when there are thousands of variables.
Stepwise	Choose this option to perform a forward stepwise method.
LAR	Choose this option to perform a least-angle regression ¹ , which begins with no effects. The parameter estimates at any step are shrunken when compared to the corresponding least squares estimates. If the model contains classification variables, then these classification variables are split. ² See the SPLIT option in the CLASS statement of SAS PROC GLMSELECT for details.
LASSO	Choose this option to add and delete parameters based on a version of ordinary least squares where the sum of the absolute regression coefficients is constrained ³ . If the model contains classification variables, then these classification variables are split. 2
Elastic Net	Choose this option to augment the data and use a LASSO fit in accordance with an Elastic Net. ⁴