The Missing Value Report

After you click OK in the launch window, the report opens to show a Commands outline and a Missing Columns report. The commands are the following:

•

Missing Value Report

•

Missing Value Clustering

•

Missing Value Snapshot

•

Multivariate Normal Imputation (Not available if you entered a Numeric column with a Nominal or Ordinal modeling type in the launch window.)

•

Multivariate SVD Imputation (Not available if you entered a Numeric column with a Nominal or Ordinal modeling type in the launch window.)

Missing Value Report

The Missing Value Report opens the Missing Columns report, which lists the name of each column and the number of missing values in that column.

Show only columns with missing

Removes columns from the list that do not have missing values.

Closes the Missing Columns report.

Select Rows

Selects the rows in the data table that contain missing values for the column(s) that you select in the Missing Columns report.

Exclude Rows

Applies the excluded row state for rows in the data table that contain missing values for the column(s) that you select in the Missing Columns report.

Color Cells

Colors the cells in the data table that contain missing values for the column(s) that you select in the Missing Columns report.

Color Rows

Colors the rows in the data table that contain missing values for the column(s) that you select in the Missing Columns report.

Missing Value Clustering

Missing Value Clustering provides a hierarchical clustering analysis of the missing data.

•

The dendrogram to the right of the plot shows clusters of missing data pattern rows. These are the rows that you would obtain by using Tables > Missing Data Pattern.

•

The dendrogram beneath the plot shows clusters of variables.

Use this report to determine if certain groups of columns tend to have similar patterns of missing values.

The rows of the plot are defined by the missing data patterns; there is a row for each pattern. The columns correspond to the variables. Each red cell indicates a group of missing values for the column listed beneath the plot. Place your cursor in a cell to see the list of values represented. Click in the plot to select missing data pattern rows. Vertical bars appear to indicate the selected patterns.

Missing Value Snapshot

The Missing Value Snapshot shows a cell plot for the missing values. The columns represent the variables. Black cells indicate a missing value. This plot is especially useful in understanding missingness for longitudinal data, where subjects may withdraw from a study before the end of the data collection period.

Multivariate Normal Imputation

The Multivariate Normal Imputation utility imputes missing values based on the multivariate normal distribution. The procedure requires that all variables have a Continuous modeling type. The algorithm uses least squares imputation. The covariance matrix is constructed using pairwise covariances. The diagonal entries (variances) are computed using all non-missing values for each variable. The off-diagonal entries for any two variables are computed using all observations that are non-missing for both variables. In cases where the covariance matrix is singular, the algorithm uses minimum norm least squares imputation based on the Moore-Penrose pseudo-inverse.

Multivariate Normal Imputation allows the option to use a shrinkage estimator for the covariances. The use of shrinkage estimators is a way of improving the estimation of the covariance matrix. For more information about shrinkage estimators, see Schafer and Strimmer (2005).

Note: If a validation column is specified, the covariance matrices are computed using observations from the Training set.

Multivariate Normal Imputation Report

The imputation report explains the results of the multivariate imputation process. Results include the following:

•

Method of imputation (either least squares or minimum-norm least squares)

•

How many values were replaced

•

Shrinkage estimator on/off

•

Factor by which the off-diagonals were scaled

•

How many rows and columns were affected

•

How many different missing value patterns there were

Once the imputation is complete, the cells corresponding to imputed values in the data table are colored in light blue. If the Missing Columns report is open, it is updated to show no missing values.

Click Undo to undo the imputation and replace the imputed data with missing values.

Multivariate SVD Imputation

The Multivariate SVD Imputation utility imputes missing values using the singular value decomposition (SVD). This utility is useful for data with hundreds or thousands of variables. Because SVD calculations do not require calculation of a covariance matrix, the SVD method is recommended for wide problems that contain large numbers of variables. The procedure requires that all variables have a Continuous modeling type.

The singular value decomposition represents a matrix of observations X as X = UDV‘, where U and V are orthogonal matrices and D is a diagonal matrix.

The SVD algorithm used by default in the Multivariate SVD Imputation utility is the sparse Lanczos method, also known as the implicitly restarted Lanczos bidiagonalization method (IRLBA). See Baglama and Reichel (2005). The algorithm does the following:

Each missing value is replaced with its column’s mean.

An SVD decomposition is performed on the matrix of observations, X.

Each cell that had a missing value is replaced by the corresponding element of the UDV‘ matrix obtained from the SVD decomposition.

Steps 2 and 3 are repeated until the SVD converges to the matrix X.

Imputation Method Window

When you click Multivariate SVD Imputation, the Imputation Method window opens to show recommended settings.

Number of Singular Vectors

Number of singular vectors that are computed and used in the imputation.

Note: It is important not to specify too many singular vectors, otherwise the SVD and the imputations do not change from iteration to iteration.

Maximum Iterations

The number of iterations used in imputing the missing values.

Show Iteration Log

Opens a Details report that shows the number of iterations and gives details on the criteria.

For large problems, a progress bar shows how many dimensions the SVD has completed. You can stop the imputation and use that number of dimensions at any time.

Multivariate SVD Imputation Report

The imputation report explains the results of the multiple imputation process.

•

Method of imputation

•

How many values were replaced

•

How many rows and columns were affected

Once the imputation is complete, the Missing Columns report is automatically shown indicating no missing values in the columns that were imputed. Imputed values are displayed in light blue.

Click Undo to undo the imputation and replace the imputed data with missing values.