Re: [R] troubleshooting data structure to run krippendorff's alpha
Hi Hallie, I tried both the "cccUst" and "cccvc" functions in the "cccrm" package. While I can get what looks like sensible statistics with the following example, I am not sure that it can be interpreted as you wish. For one thing, it assumes that the concordance will be the same on all variables. I was not able to get a statistic on each variable separately. Perhaps someone who is more familiar with the package can offer better advice. Also, check the "hkdf" data frame to ensure that it looks like your data. hkdf<-data.frame(rater=rep(1:3,each=24),occasion=rep(rep(1:4,each=6),3), var=rep(rep(1:6,4),3),obs=runif(72)) library(cccrm) cccUst(hkdf,"obs","rater","occasion") cccvc(hkdf,"obs","rater","occasion") Jim On Wed, Jan 30, 2019 at 1:01 PM Jim Lemon wrote: > > Hi Hallie, > If I understand your email correctly, you have four repeated > observations by the three raters of the same six variables. This is a > tougher problem and I can't solve it at the moment. I'll return to > this later and see if I can offer a solution. > > > Jim __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] troubleshooting data structure to run krippendorff's alpha
Hi Hallie, If I understand your email correctly, you have four repeated observations by the three raters of the same six variables. This is a tougher problem and I can't solve it at the moment. I'll return to this later and see if I can offer a solution. Jim On Wed, Jan 30, 2019 at 3:56 AM Hallie Kamesch wrote: > > Thank you Jim, for the code, and thank you Jeff for the tutorial PDF. I've > read through the sections and I appreciate the help. > I'm in way over my head - I don't even understand enough of the vocabulary to > ask my question correctly. > Jim, in your code, I ended up with an entry of 4 observations of 6 variables. > I understand how that happened now since I read your code - that helped very > much. > My only problem, that I can't figure out, is how to make it so I have 3 > raters with 4 observations of 6 variables. > I really am trying to educate myself enough to not waste your time: I've > ?data.frame, ?sample, ?matrix, ?$names, ?attributes, etc... I read the > sections in Jeff's PDF, and the tutorials on datamentor, I'm just not finding > how to do this. I'm sorry this is such a newbie question. > thank you for your time, > hallie > __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] troubleshooting data structure to run krippendorff's alpha
Thank you Jim, for the code, and thank you Jeff for the tutorial PDF. I've
read through the sections and I appreciate the help.
I'm in way over my head - I don't even understand enough of the vocabulary
to ask my question correctly.
Jim, in your code, I ended up with an entry of 4 observations of 6
variables. I understand how that happened now since I read your code - that
helped very much.
My only problem, that I can't figure out, is how to make it so I have 3
raters with 4 observations of 6 variables.
I really am trying to educate myself enough to not waste your time: I've
?data.frame, ?sample, ?matrix, ?$names, ?attributes, etc... I read the
sections in Jeff's PDF, and the tutorials on datamentor, I'm just not
finding how to do this. I'm sorry this is such a newbie question.
thank you for your time,
hallie
On Mon, Jan 28, 2019 at 1:21 PM Hallie Kamesch
wrote:
> Hi all,
> Thank you for your responses. You are correct that it is not a matrix. I
> used the incorrect term.
> I meant I put my data in a spreadsheet with three rows and 24 columns.
>
> Sent from my iPhone
>
> > On Jan 28, 2019, at 3:36 AM, Jim Lemon wrote:
> >
> > Hi Halllie,
> > As Jeff noted, a data frame is not a matrix (it is a variety of list),
> > so that looks like your problem.
> >
> >
> hkdf<-data.frame(sample(3:5,4,TRUE),sample(1:3,4,TRUE),sample(2:4,4,TRUE),
> > sample(3:5,4,TRUE),sample(1:3,4,TRUE),sample(2:4,4,TRUE))
> > library(irr)
> > kripp.alpha(hkdf)
> > kripp.alpha(as.matrix(hkdf))
> >
> > Jim
> >
> >> On Mon, Jan 28, 2019 at 6:04 PM Hallie Kamesch <
> hallie.kame...@gmail.com> wrote:
> >>
> >> Hi -
> >> I'm trying to run Krippendorff's alpha for data consisting of 4 subjects
> >> rated on 6 events each by three raters. The ratings are interval ratio
> >> scale data.
> >>
> >> I've rearranged my data into a 3 x 24 of ratersXevents. (per this
> >> discussion on CrossValidated: (
> >>
> https://stats.stackexchange.com/questions/255164/inter-rater-reliability-for-binomial-repeated-ratings-from-two-or-more-raters/256144#256144
> )
> >> ).
> >>
> >> This is the code I've used:
> >> library(irr)
> >> dat <- read.csv(file.choose(), header = TRUE)
> >> head(dat)
> >> kripp.alpha(dat, method=c("ratio"))
> >> error message: Error in sort.list(y) : 'x' must be atomic for
> >> 'sort.list'
> >> Have you called 'sort' on a list?
> >> kripp.alpha(dat,"ratio")
> >> error message: Error in sort.list(y) : 'x' must be atomic for
> >> 'sort.list'
> >> Have you called 'sort' on a list?
> >>
> >> I read rhelp on sort, but I'm still confused. Please help!
> >> Thank you!
> >>
> >> PS
> >> I arranged my data in that matrix based upon this comment and response
> from
> >> the CrossValidated posting forum (
> >>
> https://stats.stackexchange.com/questions/255164/inter-rater-reliability-for-binomial-repeated-ratings-from-two-or-more-raters/256144#256144
> ),
> >> but my question above was rejected there.
> >>
> >>[[alternative HTML version deleted]]
> >>
> >> __
> >> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> >> https://stat.ethz.ch/mailman/listinfo/r-help
> >> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> >> and provide commented, minimal, self-contained, reproducible code.
>
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] troubleshooting data structure to run krippendorff's alpha
Hi all,
Thank you for your responses. You are correct that it is not a matrix. I used
the incorrect term.
I meant I put my data in a spreadsheet with three rows and 24 columns.
Sent from my iPhone
> On Jan 28, 2019, at 3:36 AM, Jim Lemon wrote:
>
> Hi Halllie,
> As Jeff noted, a data frame is not a matrix (it is a variety of list),
> so that looks like your problem.
>
> hkdf<-data.frame(sample(3:5,4,TRUE),sample(1:3,4,TRUE),sample(2:4,4,TRUE),
> sample(3:5,4,TRUE),sample(1:3,4,TRUE),sample(2:4,4,TRUE))
> library(irr)
> kripp.alpha(hkdf)
> kripp.alpha(as.matrix(hkdf))
>
> Jim
>
>> On Mon, Jan 28, 2019 at 6:04 PM Hallie Kamesch
>> wrote:
>>
>> Hi -
>> I'm trying to run Krippendorff's alpha for data consisting of 4 subjects
>> rated on 6 events each by three raters. The ratings are interval ratio
>> scale data.
>>
>> I've rearranged my data into a 3 x 24 of ratersXevents. (per this
>> discussion on CrossValidated: (
>> https://stats.stackexchange.com/questions/255164/inter-rater-reliability-for-binomial-repeated-ratings-from-two-or-more-raters/256144#256144)
>> ).
>>
>> This is the code I've used:
>> library(irr)
>> dat <- read.csv(file.choose(), header = TRUE)
>> head(dat)
>> kripp.alpha(dat, method=c("ratio"))
>> error message: Error in sort.list(y) : 'x' must be atomic for
>> 'sort.list'
>> Have you called 'sort' on a list?
>> kripp.alpha(dat,"ratio")
>> error message: Error in sort.list(y) : 'x' must be atomic for
>> 'sort.list'
>> Have you called 'sort' on a list?
>>
>> I read rhelp on sort, but I'm still confused. Please help!
>> Thank you!
>>
>> PS
>> I arranged my data in that matrix based upon this comment and response from
>> the CrossValidated posting forum (
>> https://stats.stackexchange.com/questions/255164/inter-rater-reliability-for-binomial-repeated-ratings-from-two-or-more-raters/256144#256144),
>> but my question above was rejected there.
>>
>>[[alternative HTML version deleted]]
>>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] troubleshooting data structure to run krippendorff's alpha
Hi Halllie,
As Jeff noted, a data frame is not a matrix (it is a variety of list),
so that looks like your problem.
hkdf<-data.frame(sample(3:5,4,TRUE),sample(1:3,4,TRUE),sample(2:4,4,TRUE),
sample(3:5,4,TRUE),sample(1:3,4,TRUE),sample(2:4,4,TRUE))
library(irr)
kripp.alpha(hkdf)
kripp.alpha(as.matrix(hkdf))
Jim
On Mon, Jan 28, 2019 at 6:04 PM Hallie Kamesch wrote:
>
> Hi -
> I'm trying to run Krippendorff's alpha for data consisting of 4 subjects
> rated on 6 events each by three raters. The ratings are interval ratio
> scale data.
>
> I've rearranged my data into a 3 x 24 of ratersXevents. (per this
> discussion on CrossValidated: (
> https://stats.stackexchange.com/questions/255164/inter-rater-reliability-for-binomial-repeated-ratings-from-two-or-more-raters/256144#256144)
> ).
>
> This is the code I've used:
> library(irr)
> dat <- read.csv(file.choose(), header = TRUE)
> head(dat)
> kripp.alpha(dat, method=c("ratio"))
> error message: Error in sort.list(y) : 'x' must be atomic for
> 'sort.list'
> Have you called 'sort' on a list?
> kripp.alpha(dat,"ratio")
> error message: Error in sort.list(y) : 'x' must be atomic for
> 'sort.list'
> Have you called 'sort' on a list?
>
> I read rhelp on sort, but I'm still confused. Please help!
> Thank you!
>
> PS
> I arranged my data in that matrix based upon this comment and response from
> the CrossValidated posting forum (
> https://stats.stackexchange.com/questions/255164/inter-rater-reliability-for-binomial-repeated-ratings-from-two-or-more-raters/256144#256144),
> but my question above was rejected there.
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] troubleshooting data structure to run krippendorff's alpha
I don't understand most of what you wrote, but when you say "matrix" you are
mistaken. A matrix is NOT the same thing as a data frame, which is what you get
when you call read.csv(). Read
RShowDoc("R-intro")
Sections 5 and 6... A data frame is a list of column vectors, while a matrix is
a vector with a dimension attribute.
You can use the as.matrix function to convert a data frame to a matrix.
On January 27, 2019 9:08:02 AM PST, Hallie Kamesch
wrote:
>Hi -
>I'm trying to run Krippendorff's alpha for data consisting of 4
>subjects
>rated on 6 events each by three raters. The ratings are interval ratio
>scale data.
>
>I've rearranged my data into a 3 x 24 of ratersXevents. (per this
>discussion on CrossValidated: (
>https://stats.stackexchange.com/questions/255164/inter-rater-reliability-for-binomial-repeated-ratings-from-two-or-more-raters/256144#256144)
>).
>
>This is the code I've used:
>library(irr)
>dat <- read.csv(file.choose(), header = TRUE)
>head(dat)
>kripp.alpha(dat, method=c("ratio"))
> error message: Error in sort.list(y) : 'x' must be atomic for
>'sort.list'
>Have you called 'sort' on a list?
>kripp.alpha(dat,"ratio")
> error message: Error in sort.list(y) : 'x' must be atomic for
>'sort.list'
>Have you called 'sort' on a list?
>
>I read rhelp on sort, but I'm still confused. Please help!
>Thank you!
>
>PS
>I arranged my data in that matrix based upon this comment and response
>from
>the CrossValidated posting forum (
>https://stats.stackexchange.com/questions/255164/inter-rater-reliability-for-binomial-repeated-ratings-from-two-or-more-raters/256144#256144),
>but my question above was rejected there.
>
> [[alternative HTML version deleted]]
>
>__
>R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.
--
Sent from my phone. Please excuse my brevity.
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[R] troubleshooting data structure to run krippendorff's alpha
Hi -
I'm trying to run Krippendorff's alpha for data consisting of 4 subjects
rated on 6 events each by three raters. The ratings are interval ratio
scale data.
I've rearranged my data into a 3 x 24 of ratersXevents. (per this
discussion on CrossValidated: (
https://stats.stackexchange.com/questions/255164/inter-rater-reliability-for-binomial-repeated-ratings-from-two-or-more-raters/256144#256144)
).
This is the code I've used:
library(irr)
dat <- read.csv(file.choose(), header = TRUE)
head(dat)
kripp.alpha(dat, method=c("ratio"))
error message: Error in sort.list(y) : 'x' must be atomic for
'sort.list'
Have you called 'sort' on a list?
kripp.alpha(dat,"ratio")
error message: Error in sort.list(y) : 'x' must be atomic for
'sort.list'
Have you called 'sort' on a list?
I read rhelp on sort, but I'm still confused. Please help!
Thank you!
PS
I arranged my data in that matrix based upon this comment and response from
the CrossValidated posting forum (
https://stats.stackexchange.com/questions/255164/inter-rater-reliability-for-binomial-repeated-ratings-from-two-or-more-raters/256144#256144),
but my question above was rejected there.
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
