[ https://issues.apache.org/jira/browse/SPARK-10578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joseph K. Bradley resolved SPARK-10578. --------------------------------------- Resolution: Fixed Assignee: Joseph K. Bradley Fix Version/s: 1.5.0 [~viirya] Yep, thanks for pointing out the right link! > pyspark.ml.classification.RandomForestClassifer does not return > `rawPrediction` column > -------------------------------------------------------------------------------------- > > Key: SPARK-10578 > URL: https://issues.apache.org/jira/browse/SPARK-10578 > Project: Spark > Issue Type: Bug > Components: ML > Affects Versions: 1.4.0, 1.4.1 > Environment: CentOS, PySpark 1.4.1, Scala 2.10 > Reporter: Karen Yin-Yee Ng > Assignee: Joseph K. Bradley > Fix For: 1.5.0 > > Original Estimate: 24h > Remaining Estimate: 24h > > To use `pyspark.ml.classification.RandomForestClassifer` with > `BinaryClassificationEvaluator`, a column called `rawPrediction` needs to be > returned by the `RandomForestClassifer`. > The PySpark documentation example of `logisticsRegression`outputs the > `rawPrediction` column but not `RandomForestClassifier`. > Therefore, one is unable to use `RandomForestClassifier` with the evaluator > nor put it in a pipeline with cross validation. > A relevant piece of code showing how to reproduce the bug can be found at: > https://gist.github.com/karenyyng/cf61ae655b032f754bfb > A relevant post due to this possible bug can also be found at: > http://apache-spark-user-list.1001560.n3.nabble.com/Issue-with-running-CrossValidator-with-RandomForestClassifier-on-dataset-td23791.html -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org