Donald Catanzaro, PhD wrote:
Hi All,
I am working on some ordinal logistic regresssions using LRM in the
Design package. My response variable has three categories (1,2,3) and
after using the creating my model and using a call to predict some
values and I wanted to use a simple .5 cut-off to classify my
probabilities into the categories.
I had two questions:
a) first, I am having trouble directly accessing the probabilities
which may have more to do with my lack of experience with R
For instance, my calls
>ologit.three.NoPerFor <- lrm(Threshold.Three ~ TECI , data=CLD,
na.action=na.pass)
>CLD$Threshold.Predict.Three.NoPerFor<- predict(ologit.three.NoPerFor,
newdata=CLD, type="fitted.ind")
>CLD$Threshold.Predict.Three.NoPerFor.Cats[CLD$Threshold.Predict.Three.NoPerFor.Threshold.Three=1
> .5] <- 1
Error: unexpected '=' in
"CLD$Threshold.Predict.Three.NoPerFor.Cats[CLD$Threshold.Predict.Three.NoPerFor.Threshold.Three="
>
>
produce an error message and it seems as R does not like the equal sign
at all. So how does one access the probabilities so I can classify them
into the categories of 1,2,3 so I can look at performance of my model ?
use == to check equality
b) which leads me to my next question. I thought that simply
calculating the percent correct off of my predictions would be
sufficient to look at performance but since my question is very much in
line with this thread
http://tolstoy.newcastle.edu.au/R/e4/help/08/04/8987.html I am not so
sure anymore. I am afraid I did not understand Frank Harrell's last
suggestion regarding improper scoring rule - can someone point me to
some internet resources that I might be able to review to see why my
approach would not be valid ?
Percent correct will give you misleading answers and is game-able. It
is also ultra-high-variance. Though not a truly proper scoring rule,
Somers' Dxy rank correlation (generalization of ROC area) is helpful.
Better still: use the log-likelihood and related quantities (deviance,
adequacy index as described in my book).
Frank
--
Frank E Harrell Jr Professor and Chair School of Medicine
Department of Biostatistics Vanderbilt University
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
