Repository: spark Updated Branches: refs/heads/master 774715d5c -> 753793bc8
[SPARK-22899][ML][STREAMING] Fix OneVsRestModel transform on streaming data failed. ## What changes were proposed in this pull request? Fix OneVsRestModel transform on streaming data failed. ## How was this patch tested? UT will be added soon, once #19979 merged. (Need a helper test method there) Author: WeichenXu <weichen...@databricks.com> Closes #20077 from WeichenXu123/fix_ovs_model_transform. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/753793bc Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/753793bc Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/753793bc Branch: refs/heads/master Commit: 753793bc84df805e519cf59f6804ab26bd02d76e Parents: 774715d Author: WeichenXu <weichen...@databricks.com> Authored: Wed Dec 27 17:31:12 2017 -0800 Committer: Joseph K. Bradley <jos...@databricks.com> Committed: Wed Dec 27 17:31:12 2017 -0800 ---------------------------------------------------------------------- .../main/scala/org/apache/spark/ml/classification/OneVsRest.scala | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/753793bc/mllib/src/main/scala/org/apache/spark/ml/classification/OneVsRest.scala ---------------------------------------------------------------------- diff --git a/mllib/src/main/scala/org/apache/spark/ml/classification/OneVsRest.scala b/mllib/src/main/scala/org/apache/spark/ml/classification/OneVsRest.scala index 3ab99b3..f04fde2 100644 --- a/mllib/src/main/scala/org/apache/spark/ml/classification/OneVsRest.scala +++ b/mllib/src/main/scala/org/apache/spark/ml/classification/OneVsRest.scala @@ -165,7 +165,7 @@ final class OneVsRestModel private[ml] ( val newDataset = dataset.withColumn(accColName, initUDF()) // persist if underlying dataset is not persistent. - val handlePersistence = dataset.storageLevel == StorageLevel.NONE + val handlePersistence = !dataset.isStreaming && dataset.storageLevel == StorageLevel.NONE if (handlePersistence) { newDataset.persist(StorageLevel.MEMORY_AND_DISK) } --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
