The Each Pair Stepwise, Newman-Keuls test compares the sample means using an iterative, stepwise procedure. At each iteration, Tukey’s HSD test is used to test the difference between two group means. For an example of this test, see Example of the Each Pair Stepwise, Newman-Keuls Test.
Caution: The Newman-Keuls test does not control the family-wise error rate. Use caution when interpreting the results of this procedure.
The following procedure is used for testing J group means:
Define the following:
J = number of groups (sorted in ascending order of group means)
N = number of observations
d = degrees of freedom, calculated as N - J
i = index of smallest group mean involved in a comparison
j = index of largest group mean involved in a comparison
k = minimum value of j in any comparison during the procedure
At the beginning of the procedure, set i = 1, j = J, and k = 2.
1. Perform Tukey’s HSD test for groups i and j, where the number of groups for finding the appropriate quantile equals j - i + 1.
– If the test is significant, groups i and j are determined to be significantly different. Decrease j by 1.
If this causes j to be less than k, then increase i by 1, set k = max(i, j) + 1, set j = J, and continue to step 2.
If this causes j to be greater than or equal to k, then continue to step 2.
– If the test is not significant, groups i and j are not determined to be significantly different. Increase i by 1, set k = j + 1, set j = J, and continue to step 2.
2. Determine whether the procedure continues or stops based on the value of k.
– If k is greater than i and k is less than or equal to J, repeat step 1.
– If k is less than or equal to i or k is greater than J, stop the procedure. Any remaining untested ranges are deemed not to be significantly different.
The quantile used for Tukey’s HSD is different for each test and is based on the number of group means between the sorted means being tested. In the Newman-Keuls report, the Smallest Quantile Considered (labeled Smallest q*) is the smallest studentized range quantile used in the above procedure divided by the square root of 2.
The test results are reported in the Connecting Letters Report.
For more information about the Newman-Keuls test, see Howell (2013).
Note: There are no mean circles added to the Comparison Circles graph when you use the Each Pair Stepwise, Newman-Keuls test. This is because each comparison has a different cut-off depending on the number of means between the two means being tested. Therefore, each circle would be a different size.