[
https://issues.apache.org/jira/browse/SPARK-6885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14644196#comment-14644196
]
Yanbo Liang commented on SPARK-6885:
------------------------------------
[~josephkb]
Thanks for your comments.
After survey I found that we have two candidate plan:
#1 We record the raw counts for each label in an Array[Double] at every
LearningNode. That is we need to implement a new class PredictionStats which
stores the "counts" array.
class PredictionStats(
val predict: Double,
val counts: Array[Double]) extends Serializable {
}
Compared with the old Predict class, we just add more prediction statistic
information.
class Predict(
val predict: Double,
val prob: Double = 0.0) extends Serializable {
}
And we need to make corresponding change to InformationGainStats and
calculatePredictionStats(), maybe need a new InformationGainStats which will
not affect the old mllib code.
#2 We only record the raw counts for each label at leaf node of LearningNode.
That is we need to implement two kinds of LearningNode (InternalLearningNode
and LeafLearningNode).
I prefer the #1, looking forward your comments.
> Decision trees: predict class probabilities
> -------------------------------------------
>
> Key: SPARK-6885
> URL: https://issues.apache.org/jira/browse/SPARK-6885
> Project: Spark
> Issue Type: Sub-task
> Components: ML
> Affects Versions: 1.3.0
> Reporter: Joseph K. Bradley
> Assignee: Yanbo Liang
>
> Under spark.ml, have DecisionTreeClassifier (currently being added) extend
> ProbabilisticClassifier.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
