[
https://issues.apache.org/jira/browse/SPARK-9478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15344950#comment-15344950
]
Seth Hendrickson commented on SPARK-9478:
-----------------------------------------
There has been a bit of confusion regarding this JIRA, I think. [~pcrenshaw]
Please do correct me if I'm wrong, but the JIRA is, in truth, for adding a
mechanism to handle imbalanced classification datasets. This could be done
through class weighting or through instance weighting, I suppose the
implementation is up for debate.
There has been potentially more confusion since an initial PR was made using
instance weighting. Now there is a PR made which adds class weighting. I think
adding instance weighting is the best approach here because it allows users to
handle imbalanced outcome classes in their data, but also adds the ability to
use instance weighting generically which has a broad range of use cases.
Additionally, it is not specific to classification. Also, this is how the other
ML algorithms have so far dealt with it and it will allow forests and trees to
conform to the same API as Logistic/Linear regression, for example.
I vote to change this JIRA title to "Add instance weights for Random Forest and
Decision Trees" and proceed accordingly, but I'm open to other opinions. If we
want to pursue class weights we can do it in a separate JIRA. And again, I have
a PR ready for this which I have not submitted because of a.) other blocking
issues and b.) Spark 2.0 QA takes review precedence for the time being.
I look forward to others' thoughts. Ping [~josephkb] (I cannot ping n-triple-a
because I don't know the JIRA username).
> Add class weights to Random Forest
> ----------------------------------
>
> Key: SPARK-9478
> URL: https://issues.apache.org/jira/browse/SPARK-9478
> Project: Spark
> Issue Type: Improvement
> Components: ML, MLlib
> Affects Versions: 1.4.1
> Reporter: Patrick Crenshaw
>
> Currently, this implementation of random forest does not support class
> weights. Class weights are important when there is imbalanced training data
> or the evaluation metric of a classifier is imbalanced (e.g. true positive
> rate at some false positive threshold).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
