Example of Rater Agreement
Use the Categorical platform to summarize data from multiple raters. In the data, each column contains ratings for a single rater.
1. Select Help > Sample Data Folder and open Attribute Gauge.jmp.
2. Select Analyze > Consumer Research > Categorical.
3. Select the Related tab.
4. Select A, B, and C, and click Rater Agreement on the Related tab.
5. Click OK.
Figure 3.33 Rater Agreement Report
The rater agreement is strong as shown by the Kappa statistics. The Kappa statistic can take on a value between 0 (no agreement) to 1.0 (perfect agreement). The details section provides 2x2 tables for each pair of raters. The Bowker test of symmetry tests the null hypothesis that cell proportions are symmetric for all pairs of cells (pij = pji for all i, j). Here, p-values for the Bowker test are all greater than 0.05, indicating no strong evidence of asymmetry between raters.