[GitHub] spark pull request #19186: [SPARK-21972][ML] Add param handlePersistence

[smurching]

Github user smurching commented on a diff in the pull request:

    https://github.com/apache/spark/pull/19186#discussion_r138137893
  
    --- Diff: 
mllib/src/main/scala/org/apache/spark/ml/classification/OneVsRest.scala ---
    @@ -163,9 +165,7 @@ final class OneVsRestModel private[ml] (
         val initUDF = udf { () => Map[Int, Double]() }
         val newDataset = dataset.withColumn(accColName, initUDF())
     
    -    // persist if underlying dataset is not persistent.
    -    val handlePersistence = dataset.rdd.getStorageLevel == 
StorageLevel.NONE
    -    if (handlePersistence) {
    +    if ($(handlePersistence)) {
    --- End diff --
    
    See comment above, we should also check that `dataset.storageLevel == 
StorageLevel.NONE` before caching `newDataset`


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org