Dear Changbin,
I want to know how to select the optimal decision threshold from the ROC
curve?
Depends on what optimal means. I think there are a bunch of different criteria
used:
- point closest to the ideal model
- point furthest from the "guessing" model
- these criteria may include costs, i.e. a FP/FN ratio != 1
- ...
More practical:
If you use ROCR: the help of the performance class explains the slots in the
object. You find there the data of the curve, incl. the thresholds.
At what threshold will give the highest accuracy?
to know that, optmize the accuracy as function of the threshold.
Remember: finding the optimal threshold from a ROC curve is a data-driven
optimization. You need to validate the resulting model with independent test
data afterwards.
--
Claudia Beleites
Dipartimento dei Materiali e delle Risorse Naturali
Università degli Studi di Trieste
Via Alfonso Valerio 6/a
I-34127 Trieste
phone: +39 0 40 5 58-37 68
email: cbelei...@units.it
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
