Make a 2×2 table (2 observers rating a sample of patients as either positive or negative for a finding) where the observed agreement is less than 50%, but Kappa is nonetheless more than 0.
If observed agreement is greater than expected agreement, kappa will be greater than 0.
To get the kappa above 0 with observed agreement less than 50%, you need an expected agreement less than 50%. One can construct such a 2×2 table by making the marginals disparate, i.e., having unbalanced disagreement. This decreases the expected agreement and leads to a higher kappa.
Here’s a simple example:
Observed agreement = (1+1)/ 5 = 40%
Expected agreement = (1/5×4 + 4/5×1)/5 = (0.8 + 0.8)/5 = 1.6/5 = 32%
Kappa = (40% − 32%) / (100% − 32%) = 0.118
Back to all Chapter 5 Problems
Back to all Chapters