Please note that predicted1 and predicted2 are two sets of predictions instead of predictors. As you can see the predictions with only two levels, 1 is for hard and 2 for soft. I need to assess which one is more accurate. Hope this is clear now. Thanks. Jin
-----Original Message----- From: David Winsemius [mailto:dwinsem...@comcast.net] Sent: Thursday, 2 June 2011 10:55 AM To: Li Jin Cc: R-help@r-project.org Subject: Re: [R] aucRoc in caret package [SEC=UNCLASSIFIED] Using AUC for discrete predictor variables with inly two levels doesn't seem very sensible. What are you planning to to with this measure? -- David. On Jun 1, 2011, at 8:47 PM, <jin...@ga.gov.au> <jin...@ga.gov.au> wrote: > Hi all, > I used the following code and data to get auc values for two sets of > predictions: > library(caret) >> table(predicted1, trainy) > trainy > hard soft > 1 27 0 > 2 11 99 >> aucRoc(roc(predicted1, trainy)) > [1] 0.5 > > >> table(predicted2, trainy) > trainy > hard soft > 1 27 2 > 2 11 97 >> aucRoc(roc(predicted2, trainy)) > [1] 0.8451621 > > predicted1: > 1 1 2 2 2 1 2 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 1 2 2 2 2 2 2 2 2 2 1 2 > 2 2 2 2 1 2 2 2 2 1 1 2 2 2 2 2 1 2 2 2 2 2 1 2 2 2 2 2 2 2 2 2 2 2 > 2 2 2 1 2 2 2 2 2 2 2 1 2 2 2 2 2 1 1 1 2 2 1 1 1 2 2 2 2 2 1 1 2 2 > 2 2 2 2 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 > > predicted2: > 1 1 2 1 2 1 2 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 1 2 2 2 2 2 2 2 2 1 1 2 > 2 2 2 2 1 2 2 2 2 1 1 2 2 2 2 2 1 2 2 2 2 2 1 2 2 2 2 2 2 2 2 2 2 2 > 2 2 2 1 2 2 2 2 2 2 2 1 2 2 2 2 2 1 1 1 2 2 1 1 1 2 2 2 2 2 1 1 2 2 > 2 2 2 2 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 > > trainy: > hard hard hard soft soft hard hard hard hard soft soft soft soft > soft soft hard soft soft soft soft soft soft hard soft soft soft > soft soft soft soft soft soft hard soft soft soft soft soft hard > soft soft soft soft hard hard soft soft soft hard soft hard soft > soft soft soft soft hard soft soft soft soft soft soft soft soft > hard soft soft soft soft soft hard soft soft soft soft soft soft > soft hard soft soft soft hard hard hard hard hard soft soft hard > hard hard soft hard soft soft soft hard hard soft soft soft soft > soft hard hard hard hard hard hard hard soft soft soft soft soft > soft soft soft soft soft soft soft soft soft soft soft hard soft > soft soft soft soft soft soft soft > Levels: hard soft > >> Sys.info() > sysname > release version nodename > "Windows" "XP" "build > 2600, Service Pack 3" "PC-60772" > machine > "x86" > > I would expect predicted1 is more accurate that the predicted2. But > the auc values show an opposite. I was wondering whether this is a > bug or I have done something wrong. Thanks for your help in advance! > > Cheers, > > Jin > ____________________________________ > Jin Li, PhD > Spatial Modeller/Computational Statistician > Marine & Coastal Environment > Geoscience Australia > GPO Box 378, Canberra, ACT 2601, Australia > > Ph: 61 (02) 6249 9899; email: > jin...@ga.gov.au<mailto:jin...@ga.gov.au> > _______________________________________ > > > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. David Winsemius, MD West Hartford, CT ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
