The Fit Response Screening Report
When you launch Response Screening through the Fit Model platform, the Fit Response Screening report contains an Effect Tests table, two plots, and a Parameter Estimates table. The plots shown are the FDR PValue Plot for Effects and the FDR Logworth by Effect Size plot. Both plots are interpreted in the same way as is the platform itself. See Response Screening Plots. If random effects are included, the report also contains a Variance Components table and a BLUP - Random Effect Predictions table.
Fitting Status
The Fitting Status report is shown if one or more of the fits does not converge. The report contains a table that shows each model and whether the model converged. If the model did not converge, there is a brief description that explains why the model did not converge.
Effect Tests Table
The Effect Tests table contains a row for each pair consisting of a Y variable and a model Effect. If you select the Robust Fit option on the launch window, the models are fit using Huber M-estimation. If you specified a By variable, an Effect Tests table is created for each level of the By variable. The table includes the following columns:
Y
The specified response columns.
Switch
(Available when Switch columns are specified in the launch.) Specifies the column that is included in the model. This identifies which model the row is describing. When the Effect column contains “Switch”, the Switch column specifies the effect.
Tip: Click on the Switch column heading to sort the Effect Tests table by Switch. This enables you to more clearly see the effects within each model tested. For multiple Ys, sort on Switch and then on Y to arrange the effects within models for each response.
Effect
The specified model effects.
F Ratio
The test statistic for a test of the Effect. This is the value found in the Effect Tests report in Least Squares Fit.
Chi Square
(Available only when a robust fit is specified in the launch.) The test statistic for a test of the Effect.
PValue
The p-value for the significance test corresponding to the FRatio. See “Effect Tests” in Fitting Linear Models for more information about Effect Tests.
Logworth
The quantity -log10(p-value). This transformation adjusts p-values to provide an appropriate scale for graphing. A value that exceeds 2 is significant at the 0.01 level (because -log10(0.01) = 2).
FDR PValue
The False Discovery Rate p-value calculated using the Benjamini-Hochberg technique. This technique adjusts the p-values to control the false discovery rate for multiple tests. For more information about the FDR correction, see Benjamini and Hochberg (1995). For more information about the false discovery rate, see Statistical Details for the Response Screening Platform or Westfall et al. (2011).
FDR Logworth
The quantity -log10(FDR PValue). This is the best statistic for plotting and assessing significance. Note that small p-values result in high FDR logworth values.
Rank Fraction
(Not shown by default.) The rank of the FDR Logworth expressed as a fraction of the number of tests. If the number of tests is m, the largest FDR logworth value has Rank Fraction 1/m, and the smallest has Rank Fraction 1. Equivalently, the Rank Fraction ranks the p-values in increasing order, as a fraction of the number of tests. The Rank Fraction is used in plotting the PValues and FDR PValues in rank order of decreasing significance.
Effect Size
Indicates the extent to which response values differ across the levels or values of X. Effect sizes are scale invariant.
– The effect size is the square root of the average sum of squares from the hypothesis test divided by a robust estimate of the response standard deviation. If the interquartile range (IQR) is nonzero and IQR > range/20, the standard deviation estimate is IQR/1.3489795. Otherwise, the sample standard deviation is used.
– When there are random effects in the model, a pseudo sum of squares is calculated for each effect (Tippey and Longneck 2016). The effect size is then calculated as before but with the pseudo sum of squares in the numerator. The pseudo sum of squares for each effect is calculated as follows:
SSpseudo = F Ratio × DFNum × MSE
– When a robust fit is specified in the launch, the effect size is calculated as the square root of (chi-square test statistic/n), where n is the number of observations.
Test DF
(Available only when there are fixed effects specified in the launch window and no random effects.) The degrees of freedom for the effect test.
DFNum
(Available only when random effects are specified in the launch window.) The numerator degrees of freedom for the effect test.
DFDen
(Available only when random effects are specified in the launch window.) The denominator degrees of freedom for the effect test. DFDen is calculated using the Kenward-Roger first order approximation. See Statistical Details for the Kackar-Harville Correction in Fitting Linear Models.
Parameter Estimates
The Parameter Estimates report provides details for the fixed effect parameters specified in the models. The Parameter Estimates report contains the following columns:
Y
The response column for the corresponding estimated parameter.
Term
The model term that corresponds to the estimated parameter. The first term is always the intercept, unless you selected the No Intercept option in the Fit Model launch window. Continuous columns that are part of higher order terms are centered by default. Nominal or ordinal effects appear with values of levels in brackets. See The Factor Models in Fitting Linear Models for information about the coding of nominal and ordinal terms.
Estimate
The parameter estimate for each term. This is the estimate of the term’s coefficient in the model.
Std Error
An estimate of the standard error for the parameter estimate.
Variance Components
The Variance Components report provides details for the variance components of the random effects that are specified in the models. The Variance Components report contains the following columns:
Y
The response column for the corresponding estimated parameter.
Random Effect
The random effect term corresponding to the estimated parameter.
Var Ratio
The ratio of the variance component for the effect to the variance component for the residual. It compares the effect’s estimated variance to the model’s estimated error variance.
Var Component
The estimate variance component for the random effect.
BLUPs - Random Effect Predictions
For each combination of Y response and random effect parameter, the Random Effects Predictions report provides an estimate known as the best linear unbiased predictor (BLUP). These estimates can be used to make conditional predictions.