You can probably average the precision recall curve
or use some ranking metrics [1].
Arnaud
[1] Mining Multi-label Data
http://lkm.fri.uni-lj.si/xaigor/slo/pedagosko/dr-ui/tsoumakas09-dmkdh.pdf
On 19 Jul 2013, at 08:56, Eustache DIEMERT <[email protected]> wrote:
> I'm no expert, but I know this paper which proposes a generalization of the
> ROC for multiclass:
>
> [Hand 2001]: A simple generalization of the area under the ROC curve to
> multiple class classification problems
>
> HTH
>
> Eustache
>
>
> 2013/7/17 Josh Wasserstein <[email protected]>
> Hi,
>
> I understand that ROC-AUC is defined for binary classification problems.. But
> is there an equivalent metric for multi-label problems? i.e. something that
> does not pick a specific threshold (precision/recall regime) for each label,
> but that returns instead a summarizing measure of the overall performance
> (i.e. some statistic that summarizes how the distribution of scores across
> all labels correlate with the ground truth).
>
> Also, say that I want to way to run GridSearchCV to optimize my own metric
> in a multi-label problem (e.g. the average AUC score across all labels). Does
> anyone have any pointers to specific steps on how to do this?
>
> Thanks,
>
> Josh
>
>
>
> ------------------------------------------------------------------------------
> See everything from the browser to the database with AppDynamics
> Get end-to-end visibility with application monitoring from AppDynamics
> Isolate bottlenecks and diagnose root cause in seconds.
> Start your free trial of AppDynamics Pro today!
> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
> _______________________________________________
> Scikit-learn-general mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
> ------------------------------------------------------------------------------
> See everything from the browser to the database with AppDynamics
> Get end-to-end visibility with application monitoring from AppDynamics
> Isolate bottlenecks and diagnose root cause in seconds.
> Start your free trial of AppDynamics Pro today!
> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk_______________________________________________
> Scikit-learn-general mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
------------------------------------------------------------------------------
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
_______________________________________________
Scikit-learn-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
