cohen.kappa(cbind(x,y)) works for me. -Peter Ehlers
On 2010-05-25 7:42, Scot W. McNary wrote:
Hi, It doesn't seem happy with non-numeric data in the data frame version. Maybe a recode would work? > x1 <- c(1, 2, 3, 1) > y1 <- c(1, 3, 3, 1) > cohen.kappa(data.frame(x = x1, y = y1)) Call: cohen.kappa1(x = x, w = w, n.obs = n.obs, alpha = alpha) Cohen Kappa and Weighted Kappa correlation coefficients and confidence boundaries lower estimate upper unweighted kappa 0.098 0.60 1.1 weighted kappa 0.601 0.86 1.1 The unweighted kappa is .60. I also tried to create the p x p table version, thinking that it would be unhappy with less than a p x p table. The example dataset produces a 3 x 2 table, so this pads it out to a 3 x 3: > x <- factor(x) > y <- factor(y, levels = c("blue", "red", "yellow")) > table(x,y) y x blue red yellow blue 1 0 0 red 0 2 0 yellow 1 0 0 > cohen.kappa(table(x,y)) Call: cohen.kappa1(x = x, w = w, n.obs = n.obs, alpha = alpha) Cohen Kappa and Weighted Kappa correlation coefficients and confidence boundaries lower estimate upper unweighted kappa 0.098 0.6 1.10 weighted kappa -0.693 0.0 0.69 Number of subjects = 4> The unweighted kappa is the same as above and as found by Jim with other packages. Scot On 5/25/2010 7:31 AM, Jim Lemon wrote:On 05/25/2010 06:01 PM, Jason Priem wrote:Hi, I've got two vectors with ratings from two coders, like this: x<-c("red", "yellow", "blue", "red") #coder number 1 y<-c("red", "blue", "blue", "red") #coder number 2 I want to find Cohen's Kappa using the wkappa function in the psych package. The only example in the docs is using a matrix, which I'm afraid I don't understand--I don't know how to get there from what I've got. Could anyone point me in the right direction? Thanks!Hi Jason, You're not alone. I tried to work out how to run this and it took a while. Both kappa2 in the "irr" package: kappa2(cbind(x,y)) and classAgreement in the "e1071" package: classAgreement(table(x,y)) produce a kappa of 0.6. I was unable to work out how to use the wkappa function in the "psych" package: wkappa(table(x,y)) The above led to a kappa of -0.8. Although wkappa is presented as a test of the reliability of ratings of nominal level data, the example uses numeric data. Jim
______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
