[ https://issues.apache.org/jira/browse/SPARK-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14621607#comment-14621607 ]
Gang Bai commented on SPARK-2401: --------------------------------- Yes, this should be based on a basic abstraction of AdaBoost or additive models, and use Pipelines API. BTW, I implemented the AdaBoost.MH algo, experimentally and in a separate repo. It was based on Spark/MLLib 1.1, so it's not with Pipelines API. It's in the repo https://github.com/BaiGang/spark_multiboost, and also I made some presentations about this work https://github.com/BaiGang/slides/tree/master/spark_multiboost. I am willing to improve it, ideally make it base on SPARK-1546 and use Pipelines API and part of mllib. > AdaBoost.MH, a multi-class multi-label classifier > ------------------------------------------------- > > Key: SPARK-2401 > URL: https://issues.apache.org/jira/browse/SPARK-2401 > Project: Spark > Issue Type: New Feature > Components: MLlib > Reporter: Gang Bai > Priority: Trivial > > Multi-class multi-label classifiers are very useful in web page profiling, > audience segmentation etc. The goal of a multi-class multi-label classifier > is to tag a sample data point with a subset of labels from a finite, > pre-specified set, e.g. tagging a visitor with a set of interests. Given a > set of L labels, a data point can be tagged with one of the 2^L possible > subsets. The main challenges in training a multi-class multi-label classifier > are the exponentially large label space. > This JIRA is created to track the effort of solving the training problem of > multi-class, multi-label classifiers by implementing AdaBoost.MH on Apache > Spark. It will not be an easy task. I will start from a basic DecisionStump > weak learner and a simple Hamming tree resembling DecisionStumps into a meta > weak learner, and the iterative boosting procedure. I will be reusing modules > of Alexander Ulanov's multi-class and multi-label metrics evaluation and > Manish Amde's decision tree/boosting/ensemble implementations. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org