Model Comparison Platform Options
The options in the Model Comparison red triangle menu depend on whether the response is continuous or categorical.
Continuous Responses
When the response is continuous, the Model Comparison red triangle menu contains the following options:
Model Averaging
Creates a new column of the arithmetic mean of the predicted values across models.
Plot Actual by Predicted
Shows or hides a scatterplot of the actual versus the predicted values. The plots for the different models are overlaid.
Plot Residual by Row
Shows or hides a plot of the residuals by row number. The plots for the different models are overlaid.
Profiler
Shows or hides a profiler for each response based on prediction formula columns in your data. The profilers have a row for each model being compared.
Categorical Responses
When the response is categorical, the Model Comparison red triangle menu contains the following options:
Model Averaging
Creates a new column of the arithmetic mean of the predicted probabilities across models.
ROC Curve
Shows or hides ROC curves for each level of the response variable. The curves for the different models are overlaid. See ROC Curve.
AUC Comparison
Shows or hides a comparison of the area under the ROC curve (AUC) from each model. The area under the curve is the indicator of the goodness of fit, where 1 is a perfect fit.
The report includes the following information:
– standard errors and confidence intervals for each AUC
– standard errors, confidence intervals, and hypothesis tests for the difference between each pair of AUCs
– an overall hypothesis test for testing whether all AUCs are equal
Precision Recall Curve
Shows or hides Precision-Recall Curve plots for each level of the response variable. The curves for the different models are overlaid in the plots. See Precision-Recall Curve.
Lift Curve
Shows or hides lift curves for each level of the response variable. The curves for the different models are overlaid. See Lift Curve.
Cum Gains Curve
Shows or hides cumulative gains curves for each level of the response variable. A cumulative gains curve is a plot of the proportion of a response level that is identified by the model against the proportion of all responses. A cumulative gains curve for a perfect model would reach 1.0 at the overall proportion of the response level. The curves for the different models are overlaid.
Confusion Matrix
Shows or hides confusion matrices for each model. A confusion matrix is a two-way classification of actual and predicted responses. Count and rate confusion matrices are shown. Separate confusion matrices are produced for each level of the Group variable.
If the response has a Profit Matrix column property, then Actual by Decision Count and Actual by Decision Rate matrices are shown to the right of the confusion matrices. For more information about these matrices, see Additional Examples of the Partition Platform.
Profiler
Shows or hides a profiler for each response based on prediction formula columns in your data. The profilers have a row for each model being compared.
Decision Threshold
(Available only for binary categorical responses.) Shows or hides Decision Thresholds reports for the training, validation, and test sets, if specified. Each report contains a graph of the distribution of fitted probabilities for each model, confusion matrices for each model, classification graphs to compare the model fits, and a table of classification accuracy metrics. See Decision Thresholds Report.