Hi Max and Andrew,
Thanks so much for your reply. Indeed I found your link last night using
steps shown below.
My first question is if the following two steps are right and AUC is 51% as
shown below.
My seond question is that currently I am using cost parameter=1 (the default
in R-SVM; http://www.stanford.edu/group/wonglab/RSVMpage/R-SVM.html). To
improve the AUC from ROC curve, can I can optimize the SVM cost function
(instead of keeping it fixed a 1) to get the mimimum LOO error in training
cross.validation and then draw the ROC from decision.values as Step2 below
using a cost parameter that gave mimimum cross.validation error in the
training data. Is that right?
Many thanks.
--
*Step 1: For obtaining ROC curve of test data I turned on prob=T option:*
svmres.prob - svm(traindx[,resrsvm$SelInd], as.factor(traindy),
decision.values = TRUE)
svmpred.prob - predict(svmres.prob, testdx[,resrsvm$SelInd],
decision.values = TRUE)
print(confusionMatrix(svmpred.prob,testdy))
Confusion Matrix and Statistics
Reference
Prediction Resistant Sensitive
Resistant 513
Sensitive3788
Accuracy : 0.6503
*Step 2: Actual ROC plot command using output from above and plot attached
as well as pdf (I am assuming the following says the AUC is 51.4):*
library(ROCR)
svm.roc - prediction(attributes(svmpred.prob)$decision.values, testdy)
svm.auc - performance(svm.roc, 'tpr', 'fpr')
aucsvm - performance(svm.roc, 'auc')
pdf(file=roc_curve_rsvm_decval.pdf)
plot(svm.auc)
print(str(aucsvm))
print(str(aucsvm))
Formal class 'performance' [package ROCR] with 6 slots
..@ x.name : chr None
..@ y.name : chr Area under the ROC curve
..@ alpha.name : chr none
..@ x.values: list()
..@ y.values:List of 1
.. ..$ : num 0.514
..@ alpha.values: list()
---
On Tue, Feb 22, 2011 at 4:23 PM, Andrew Ziem az...@us.ci.org wrote:
In addition's to Max's suggestion about caret, look at ROCR which
visualizes ROC charts for any binary classifier. I have an example of
e1071::SVN and ROCR here
https://heuristically.wordpress.com/2009/12/23/compare-performance-machine-learning-classifiers-r/
-Original Message-
From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org]
On Behalf Of Angel Russo
Sent: Monday, February 21, 2011 3:34 PM
To: r-help@r-project.org
Subject: [R] ROC from R-SVM?
*Hi,
*Does anyone know how can I show an *ROC curve for R-SVM*? I understand in
R-SVM we are not optimizing over SVM cost parameter. Any example ROC for
R-SVM code or guidance can be really useful.
Thanks, Angel.
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
