Example of Rater Agreement
This example uses the Attribute Gauge.jmp sample data, which has the ratings (0/1) from three operators rating 50 parts three times.
1. Select Help > Sample Data Library and open Attribute Gauge.jmp.
2. Select Analyze > Consumer Research > Categorical.
3. Select the Related tab.
4. Select A, B, and C, and click Rater Agreement on the Related tab.
5. Click OK.
Figure 3.33 Rater Agreement Report
The rater agreement is strong as shown by the Kappa statistics. The Kappa statistic can take on a value between 0 (no agreement) to 1.0 (perfect agreement). The details section provides 2x2 tables for each pair of raters. The Bowker test of symmetry tests the null hypothesis that cell proportions are symmetric for all pairs of cells (pij = pji for all i, j). Here, p-values for the Bowker test are all greater than 0.05, indicating no strong evidence of asymmetry between raters.