Model Screening Platform Options
The Model Screening red triangle menu contains the following options:
Plot Actual by Predicted
(Available only for continuous responses.) Shows or hides a report of plots that overlay the actual by predicted points from all of the model fits. There are separate plots for the Training, Validation, and Test sets. The report contains a Methods legend with check boxes that enable you to specify which fits appear on the plots based on the method, fold, inner fold, and trial combination. Click the Deselect All button at the bottom of the legend to remove all model fits from the plots.
Notes:
– If you select a subset of models from the Training, Validation, or Test reports and then select Plot Actual by Predicted, only the plots for the selected models are shown.
– The actual by predicted plots are not supported by all of the modeling platforms. A list of unsupported platforms is shown below the plots.
– This option is not available if Remove Live Reports is selected in the launch window.
Profiler
Shows or hides prediction profilers for each type of model fit. See “Profiler” in Profilers. If any type of folded crossvalidation is specified in the launch window, the selected model in the prediction profiler uses the fold, inner fold, and trial combination that corresponds to the final model. The final model is the model that produces the highest weighted average RSquare. The weight average RSquare is the average, weighted by number of observations, of the training RSquare, validation RSquare, and when a test set is used, the test RSquare.
Notes:
– Confidence intervals are shown in prediction profilers for Standard Least Squares models and Generalized Regression models.
– Prediction profilers are not available for K Nearest Neighbors models.
– This option is not available if Remove Live Reports is selected in the launch window.
ROC Curve
(Available only for categorical responses.) Shows or hides a report of plots that overlay the ROC curves for all of the model fits. There are separate plots for the Training, Validation, and Test sets. The report contains legends with check boxes that enable you to specify which fits appear on the plots. Click the Deselect All button at the bottom of the legend to remove all model fits from the plots. See “ROC Curve”.
Notes:
– If you select a subset of models from the Training, Validation, or Test reports and then select ROC Curves, only the plots for the selected models are shown.
– The ROC plots are not supported by all of the modeling platforms. A list of unsupported platforms is shown below the plots.
– This option is not available if the Remove Live Reports option is selected in the launch window.
Precision Recall Curve
(Available only for categorical responses.) Shows or hides a report of plots that overlay the precision-recall curves for all of the model fits. There are separate plots for the Training, Validation, and Test sets. The report contains legends with check boxes that enable you to specify which fits appear on the plots. Click the Deselect All button at the bottom of the legend to remove all model fits from the plots. See “Precision-Recall Curve”.
Notes:
– The Precision-Recall Curve plots are not supported by all of the modeling platforms. A list of unsupported platforms is shown below the plots.
– This option is not available if the Remove Live Reports option is selected in the launch window.
Show Profit
(Available only for binary categorical responses that have a Profit Matrix column property.) Shows or hides the expected profit for each model.
Decision Threshold
(Available only for binary categorical responses.) Shows or hides Decision Thresholds reports for the training, validation, and test sets, if specified. Each report contains a graph of the distribution of fitted probabilities for each model, confusion matrices for each model, and classification graphs to adjust thresholds and compare the model fits. See Decision Thresholds Report.
Notes:
– If you select a subset of models from the Training, Validation, or Test reports and then select Decision Threshold, only the information for the selected models is shown.
– If you specified any type of folded crossvalidation in the launch window, the probabilities shown in the Decision Threshold report are the averaged probabilities over all of the folds.
Save Results Table
Saves the information in the Validation report to a new data table titled Model Screening Statistics Validation Set. If there is a test set, such as when the Nested Crossvalidation option is specified, the information in the Test report is also saved to a new data table titled Model Screening Statistics Test Set.
Save KFold Results Table
(Available only if a Crossvalidation option is specified in the launch window.) Saves the information in the Summary Across the Folds report to a new data table titled KFold Results.
Save Prediction Formulas
(Available only if the live reports are included in the model report.) Saves the prediction formulas for the selected models to the data table. If you select a model in the Summary Across the Folds report and any type of folded crossvalidation is specified in the launch window, the prediction formula is for the fold, inner fold, and trial combination that corresponds to the final model. The final model is the model that produces the highest weighted average RSquare. The weight average RSquare is the average, weighted by number of observations, of the training RSquare, validation RSquare, and when a test set is used, the test RSquare.
Notes:
– A prediction formula is not saved for Stepwise models.
– A prediction formula is not saved for SVM models when the response is ordinal.
– This option is not available if the Remove Live Reports option is selected in the launch window.
Save Folded Prediction Formula
(Available only if folded crossvalidation methods are specified in the launch window or a KFold validation column is used.) Saves new prediction columns for the selected models to the original data table. For each selected model, prediction formula columns for each fold are added, as well as a final leak-free prediction formula column. Leaking occurs when the data used to validate model performance is also used to build the model. Therefore, for each row, the final leak-free prediction formula avoids using model fits that were trained using that row. The final prediction column contains the model name.
Caution: If no models are selected in the Summary Across the Folds table, prediction columns are saved for all models.
Notes:
– Prediction formulas are not saved for Stepwise models.
– Prediction formulas are not saved for SVM models when the response is ordinal.
– This option is not available if the Remove Live Reports option is selected in the launch window.
Optional Reports
Shows a submenu of additional report options.
Elapsed Time
Shows or hides a report that contains the total elapsed time that was spent fitting each type of model.
Cardinality of Predictors
Shows or hides a report of the number of levels and how many parameters are used in the linear model fit for each categorical predictor.
See “Local Data Filters in JMP Reports”, “Redo Menus in JMP Reports”, “Group Platform”, and “Save Script Menus in JMP Reports” in Using JMP for more information about the following options:
Local Data Filter
Shows or hides the local data filter that enables you to filter the data used in a specific report.
Redo
Contains options that enable you to repeat or relaunch the analysis. In platforms that support the feature, the Automatic Recalc option immediately reflects the changes that you make to the data table in the corresponding report window.
Platform Preferences
Contains options that enable you to view the current platform preferences or update the platform preferences to match the settings in the current JMP report.
Save Script
Contains options that enable you to save a script that reproduces the report to several destinations.
Save By-Group Script
Contains options that enable you to save a script that reproduces the platform report for all levels of a By variable to several destinations. Available only when a By variable is specified in the launch window.
Note: Additional options for this platform are available through scripting. Open the Scripting Index under the Help menu. In the Scripting Index, you can also find examples for scripting the options that are described in this section.