In the Oneway platform, there are multiple options for comparing means. This section covers options for the mean comparison reports as well as details about each comparison method.
In the Oneway platform, the Means Comparisons red triangle menu contains the following options:
Difference Matrix
Shows or hides a matrix of all pairwise differences between group means.
Confidence Quantile
Shows or hides the critical value(s) and significance level (α) used for the means comparison procedure.
Tip: Use the Set α Level option in the Oneway Analysis red triangle menu to change the significance level.
LSD Threshold Matrix
(Not available for the Each Pair Stepwise option.) Shows or hides a matrix of pairwise differences of means minus the least significant difference for those means. A positive value indicates a pair of means that are significantly different. For Hsu’s MCB test, there are two LSD matrices: one for the comparison to the minimum and one for the comparison to the maximum. For Dunnett’s test, there are options for a lower or upper threshold matrix; these options correspond to one-sided tests.
Connecting Letters Report
(Available only for the Each Pair, All Pairs, and Each Pair Stepwise options.) Shows or hides the traditional letter-coded report where means that do not share a letter are significantly different.
Ordered Differences Report
(Available only for the Each Pair and All Pairs options.) Shows or hides all pairwise positive differences, standard error of the difference, confidence intervals, p-values, and a plot of the magnitude of the difference with confidence intervals. The p-value corresponds to the hypothesis of equal means.
Detailed Comparisons Report
(Available only for the Each Pairs option.) Shows or hides a detailed report for each comparison. Each comparison shows the difference between the levels, standard error of the difference, confidence intervals, t-ratios, p-values, and degrees of freedom. A plot illustrating the comparison appears on the right of each report.
Note: The standard error of the differences is the pooled standard error based on the MSE and the sample size for each pair.
In the Oneway platform, use the Each Pair, Student’s t option to show Fisher’s least significant difference (LSD) test based on Student’s t test to test for differences across each pair of group levels. For an example of this test, see Example of the Each Pair, Student’s t Test.
In the Oneway platform, use the All Pairs, Tukey HSD option to test all pairs of differences. This method protects the overall significance level for the tests of all combinations of pairs. The HSD intervals are wider than the Student’s t pairwise LSD intervals. Compared to the comparison circles for the Each Pair comparisons, the circles for the Tukey HSD comparisons are larger and the differences between pairs of means are less significant.
In the Confidence Quantile table, the q* statistic is calculated as q* = (1/sqrt(2)) * q where q is the Alpha percentile of the studentized range distribution. For an example of this test, see Example of the All Pairs, Tukey HSD Test.
The All Pairs, Tukey HSD option performs the Tukey or Tukey-Kramer HSD (honestly significant difference) test (Tukey 1953; Kramer 1956). This test is an exact alpha-level test if the sample sizes are the same, and conservative if the sample sizes are different (Hayter 1984).
In the Oneway platform, use the With Best, Hsu MCB option to test whether the mean for a given level exceeds the maximum mean of the remaining levels, or is smaller than the minimum mean of the remaining levels. See Hsu (1996). For an example of this test, see Example of the With Best, Hsu MCB Test.
The quantiles for the Hsu MCB test vary by the level of the categorical variable. Unless the sample sizes are equal across levels, the comparison circle technique is not exact. The radius of a comparison circle is given by the standard error of the level multiplied by the largest quantile value. Use the p-values of the tests to obtain precise assessments of significant differences. See Comparison with Max and Min.
The With Best, Hsu MCB option tests performs the Hsu MCB test (Hsu 1996; Hsu 1981).
Note: Means that are not regarded as the maximum or the minimum by MCB are also the means that are not contained in the selected subset of Gupta (1965) of potential maximums or minimum means.
In addition to tables that are common to all of the available means comparisons methods, the Hsu’s MCB report contains a Comparison with Max and Min table. This table contains the following values:
Level
The level of the categorical variable.
with Max p-Value
The p-value for the test that the mean of the level exceeds the maximum mean of the remaining levels. Use the tests in this column to screen out levels whose means are significantly smaller than or equal to the (unknown) largest true mean.
with Min p-Value
The p-value for the test that the mean of the level is smaller than the minimum mean of the remaining levels. Use the tests in this column to screen out levels whose means are significantly greater than or equal to the (unknown) smallest true mean.
In the Oneway platform, use the With Control, Dunnett’s option to compare group means to a control group. One-sided and two-sided comparisons are available. The Dunnett’s least significant differences are between those of the Student’s t and Tukey-Kramer tests, as they are sized on an intermediate number of comparisons. For an example of this test, see Example of the With Control, Dunnett’s Test. For more information about Dunnett’s test, see Dunnett (1955).
In the Oneway platform, use the Each Pair Stepwise, Newman-Keuls option to compare the group means using an iterative, stepwise procedure. At each iteration, Tukey’s HSD test is used to test the difference between two group means. For an example of this test, see Example of the Each Pair Stepwise, Newman-Keuls Test.
The Each Pair Stepwise, Newman-Keuls option tests whether there are differences between the means using the Studentized range test in a stepwise procedure. This is the Newman-Keuls or Student-Newman-Keuls method (Keuls, 1952). This test is less conservative than a Tukey HSD test.
Caution: The Newman-Keuls test does not control the familywise error rate. Use caution when interpreting the results of this procedure.
The following procedure is used for testing J group means:
Define the following:
J = number of groups (sorted in ascending order of group means)
N = number of observations
d = degrees of freedom, calculated as N - J
i = index of smallest group mean involved in a comparison
j = index of largest group mean involved in a comparison
k = minimum value of j in any comparison during the procedure
At the beginning of the procedure, set i = 1, j = J, and k = 2.
1. Perform Tukey’s HSD test for groups i and j, where the number of groups for finding the appropriate quantile equals j - i + 1.
– If the test is significant, groups i and j are determined to be significantly different. Decrease j by 1.
If this causes j to be less than k, then increase i by 1, set k = max(i, j) + 1, set j = J, and continue to step 2.
If this causes j to be greater than or equal to k, then continue to step 2.
– If the test is not significant, groups i and j are not determined to be significantly different. Increase i by 1, set k = j + 1, set j = J, and continue to step 2.
2. Determine whether the procedure continues or stops based on the value of k.
– If k is greater than i and k is less than or equal to J, repeat step 1.
– If k is less than or equal to i or k is greater than J, stop the procedure. Any remaining untested ranges are deemed not to be significantly different.
The quantile used for Tukey’s HSD is different for each test and is based on the number of group means between the sorted means being tested. In the Newman-Keuls report, the Smallest Quantile Considered (labeled Smallest q*) is the smallest studentized range quantile used in the above procedure divided by the square root of 2.
The test results are reported in the Connecting Letters Report.
For more information about the Newman-Keuls test, see Howell (2013).
Note: There are no mean circles added to the Comparison Circles graph when you use the Each Pair Stepwise, Newman-Keuls test. This is because the comparison circles would vary in size at each step of the procedure.