[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22790 thanks, merging to master/2.4! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22790 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98036/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22790 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22790 **[Test build #98036 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98036/testReport)** for PR 22790 at commit [`0decb23`](https://github.com/apache/spark/commit/0decb235cc31200cd7102daddf5c2f527c04e234). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/22790 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22790 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22790 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/4493/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22790 **[Test build #98036 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98036/testReport)** for PR 22790 at commit [`0decb23`](https://github.com/apache/spark/commit/0decb235cc31200cd7102daddf5c2f527c04e234). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/22790 LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22790 Is this ready to go? We are going to have another RC, and would be good to include it. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/22790 Agree with @mengxr about this not being a blocker. Please notice this is a problem only for the `mllib` API, for which we can also avoid supporting new features AFAIK. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22790 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22790 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97942/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22790 **[Test build #97942 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97942/testReport)** for PR 22790 at commit [`29c9ca0`](https://github.com/apache/spark/commit/29c9ca0099bff199904a2ab7c528c5186d631671). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22790 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22790 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/4416/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user huaxingao commented on the issue:
https://github.com/apache/spark/pull/22790
I added a regression test in
```org.apache.spark.mllib.clustering.BisectingKMeansSuite```
I could add the following test in ml package.
```
test("SPARK-25793") {
val bisectingKMeans = new BisectingKMeans()
bisectingKMeans.setDistanceMeasure(DistanceMeasure.COSINE)
val readBisectingKMeans = testDefaultReadWrite(bisectingKMeans)
assert(bisectingKMeans.distanceMeasure ===
readBisectingKMeans.distanceMeasure)
}
```
But the bug doesn't really affect the above test. With the bug, even though
mllib ```BisectingKMeansModel.load``` will call V1_0 load and gives a model
with default value of distanceMeasure, in ml package,
```BisectingKMeansModelReader.load``` will call
```metadata.getAndSetParams(model)``` which will set the distanceMeasure to the
correct value (DistanceMeasure.COSINE).
```
override def load(path: String): BisectingKMeansModel = {
val metadata = DefaultParamsReader.loadMetadata(path, sc, className)
val dataPath = new Path(path, "data").toString
val mllibModel = MLlibBisectingKMeansModel.load(sc, dataPath)
val model = new BisectingKMeansModel(metadata.uid, mllibModel)
metadata.getAndSetParams(model)
model
}
```
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22790 **[Test build #97942 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97942/testReport)** for PR 22790 at commit [`29c9ca0`](https://github.com/apache/spark/commit/29c9ca0099bff199904a2ab7c528c5186d631671). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22790 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22790 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97928/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22790 **[Test build #97928 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97928/testReport)** for PR 22790 at commit [`77902bc`](https://github.com/apache/spark/commit/77902bc3a08d1397c1b69b68ad7aecaf1038defb). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/22790 This shouldn't block 2.4.0 release. Based on the code, it doesn't introduce regression to existing features (just using V1 format and ignore trainingCost and distanceMeasure). Correctness issue occurs only when someone uses a non-default distanceMeasure and then save/load. Could someone help confirm? If current vote passes, we can list it as an known issue in the release notes and fix it in 2.4.1. If other blockers show up, we fix it before RC5. Btw, this PR needs a regression test in order to merge. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22790 **[Test build #97928 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97928/testReport)** for PR 22790 at commit [`77902bc`](https://github.com/apache/spark/commit/77902bc3a08d1397c1b69b68ad7aecaf1038defb). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22790 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22790 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/4406/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user huaxingao commented on the issue: https://github.com/apache/spark/pull/22790 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22790 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22790 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97904/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22790 **[Test build #97904 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97904/testReport)** for PR 22790 at commit [`77902bc`](https://github.com/apache/spark/commit/77902bc3a08d1397c1b69b68ad7aecaf1038defb). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22790 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22790 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/4394/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22790 **[Test build #97904 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97904/testReport)** for PR 22790 at commit [`77902bc`](https://github.com/apache/spark/commit/77902bc3a08d1397c1b69b68ad7aecaf1038defb). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22790 cc @mengxr @WeichenXu123 how serious is it? shall we treat it as a blocker? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22790 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97882/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22790 **[Test build #97882 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97882/testReport)** for PR 22790 at commit [`d599f83`](https://github.com/apache/spark/commit/d599f837f15a188f4e995d3088f9cd174a4dc1cc). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22790 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22790 **[Test build #97882 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97882/testReport)** for PR 22790 at commit [`d599f83`](https://github.com/apache/spark/commit/d599f837f15a188f4e995d3088f9cd174a4dc1cc). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22790 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22790 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/4376/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22790 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22790 **[Test build #97794 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97794/testReport)** for PR 22790 at commit [`d599f83`](https://github.com/apache/spark/commit/d599f837f15a188f4e995d3088f9cd174a4dc1cc). * This patch **fails PySpark unit tests**. * This patch **does not merge cleanly**. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22790 **[Test build #97824 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97824/testReport)** for PR 22790 at commit [`d599f83`](https://github.com/apache/spark/commit/d599f837f15a188f4e995d3088f9cd174a4dc1cc). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22790: [SPARK-25793][ML]call SaveLoadV2_0.load for classNameV2_...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22790 **[Test build #97756 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97756/testReport)** for PR 22790 at commit [`d599f83`](https://github.com/apache/spark/commit/d599f837f15a188f4e995d3088f9cd174a4dc1cc). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
