[jira] [Commented] (SPARK-10578) pyspark.ml.classification.RandomForestClassifer does not return `rawPrediction` column
[ https://issues.apache.org/jira/browse/SPARK-10578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14743831#comment-14743831 ] Karen Yin-Yee Ng commented on SPARK-10578: -- Thanks [~josephkb] and [~viirya] for the quick response. > pyspark.ml.classification.RandomForestClassifer does not return > `rawPrediction` column > -- > > Key: SPARK-10578 > URL: https://issues.apache.org/jira/browse/SPARK-10578 > Project: Spark > Issue Type: Bug > Components: ML >Affects Versions: 1.4.0, 1.4.1 > Environment: CentOS, PySpark 1.4.1, Scala 2.10 >Reporter: Karen Yin-Yee Ng >Assignee: Joseph K. Bradley > Fix For: 1.5.0 > > Original Estimate: 24h > Remaining Estimate: 24h > > To use `pyspark.ml.classification.RandomForestClassifer` with > `BinaryClassificationEvaluator`, a column called `rawPrediction` needs to be > returned by the `RandomForestClassifer`. > The PySpark documentation example of `logisticsRegression`outputs the > `rawPrediction` column but not `RandomForestClassifier`. > Therefore, one is unable to use `RandomForestClassifier` with the evaluator > nor put it in a pipeline with cross validation. > A relevant piece of code showing how to reproduce the bug can be found at: > https://gist.github.com/karenyyng/cf61ae655b032f754bfb > A relevant post due to this possible bug can also be found at: > http://apache-spark-user-list.1001560.n3.nabble.com/Issue-with-running-CrossValidator-with-RandomForestClassifier-on-dataset-td23791.html -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-10578) pyspark.ml.classification.RandomForestClassifer does not return `rawPrediction` column
[ https://issues.apache.org/jira/browse/SPARK-10578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14743029#comment-14743029 ] Liang-Chi Hsieh commented on SPARK-10578: - Hi Karen, I think these columns are added to RandomForestClassifier in SPARK-9447 by the PR https://github.com/apache/spark/pull/7903. > pyspark.ml.classification.RandomForestClassifer does not return > `rawPrediction` column > -- > > Key: SPARK-10578 > URL: https://issues.apache.org/jira/browse/SPARK-10578 > Project: Spark > Issue Type: Bug > Components: ML >Affects Versions: 1.4.0, 1.4.1 > Environment: CentOS, PySpark 1.4.1, Scala 2.10 >Reporter: Karen Yin-Yee Ng > Original Estimate: 24h > Remaining Estimate: 24h > > To use `pyspark.ml.classification.RandomForestClassifer` with > `BinaryClassificationEvaluator`, a column called `rawPrediction` needs to be > returned by the `RandomForestClassifer`. > The PySpark documentation example of `logisticsRegression`outputs the > `rawPrediction` column but not `RandomForestClassifier`. > Therefore, one is unable to use `RandomForestClassifier` with the evaluator > nor put it in a pipeline with cross validation. > A relevant piece of code showing how to reproduce the bug can be found at: > https://gist.github.com/karenyyng/cf61ae655b032f754bfb > A relevant post due to this possible bug can also be found at: > http://apache-spark-user-list.1001560.n3.nabble.com/Issue-with-running-CrossValidator-with-RandomForestClassifier-on-dataset-td23791.html -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
