[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12683 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user GayathriMurali commented on a diff in the pull request:
https://github.com/apache/spark/pull/12683#discussion_r61481184
--- Diff: R/pkg/inst/tests/testthat/test_mllib.R ---
@@ -71,7 +71,25 @@ test_that("glm and predict", {
data = iris, family = poisson(link = identity)), iris))
expect_true(all(abs(rVals - vals) < 1e-6), rVals - vals)
- # Test stats::predict is working
+ # Test model save/load
+ modelPath <- tempfile(pattern = "GLM", fileext = ".tmp")
+ ml.save(model, modelPath)
+ expect_error(ml.save(model, modelPath))
+ ml.save(model, modelPath, overwrite = TRUE)
+ m2 <- ml.load(modelPath)
+ s2 <- summary(m2)
+ expect_equal(s$rCoefficients, s2$rCoefficients)
--- End diff --
I agree. Should we do this test for all Gaussian, Poisson and Binomial
family. I am assuming doing for either one of them should be sufficient?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user yanboliang commented on a diff in the pull request:
https://github.com/apache/spark/pull/12683#discussion_r61437879
--- Diff: R/pkg/inst/tests/testthat/test_mllib.R ---
@@ -71,7 +71,25 @@ test_that("glm and predict", {
data = iris, family = poisson(link = identity)), iris))
expect_true(all(abs(rVals - vals) < 1e-6), rVals - vals)
- # Test stats::predict is working
+ # Test model save/load
+ modelPath <- tempfile(pattern = "GLM", fileext = ".tmp")
+ ml.save(model, modelPath)
+ expect_error(ml.save(model, modelPath))
+ ml.save(model, modelPath, overwrite = TRUE)
+ m2 <- ml.load(modelPath)
+ s2 <- summary(m2)
+ expect_equal(s$rCoefficients, s2$rCoefficients)
--- End diff --
And further more, we can have a separate test case named "glm save/load"
which can test the equality of both the model and summary.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user yanboliang commented on a diff in the pull request:
https://github.com/apache/spark/pull/12683#discussion_r61437435
--- Diff: R/pkg/inst/tests/testthat/test_mllib.R ---
@@ -71,7 +71,25 @@ test_that("glm and predict", {
data = iris, family = poisson(link = identity)), iris))
expect_true(all(abs(rVals - vals) < 1e-6), rVals - vals)
- # Test stats::predict is working
+ # Test model save/load
+ modelPath <- tempfile(pattern = "GLM", fileext = ".tmp")
+ ml.save(model, modelPath)
+ expect_error(ml.save(model, modelPath))
+ ml.save(model, modelPath, overwrite = TRUE)
+ m2 <- ml.load(modelPath)
+ s2 <- summary(m2)
+ expect_equal(s$rCoefficients, s2$rCoefficients)
--- End diff --
There is no member variable ```rCoefficients``` in
```summary.GeneralizedLinearRegressionModel```, it should be
```coefficients```. Please refer L89-L98 in test_mllib.R, and the variables in
the following line has the same issues.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215315114 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57218/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215315097 **[Test build #57218 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57218/consoleFull)** for PR 12683 at commit [`6650890`](https://github.com/apache/spark/commit/665089051aef4dd4ac189eed329ee55d3e8df9e3). * This patch **fails SparkR unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215315113 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215310647 **[Test build #57218 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57218/consoleFull)** for PR 12683 at commit [`6650890`](https://github.com/apache/spark/commit/665089051aef4dd4ac189eed329ee55d3e8df9e3). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user yanboliang commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215308071 @GayathriMurali You should modify [```RWrappers.load```](https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/ml/r/RWrappers.scala#L44) at Scala side to make it support loading GLM wrapper. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215307044 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215307026 **[Test build #57211 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57211/consoleFull)** for PR 12683 at commit [`55523f7`](https://github.com/apache/spark/commit/55523f7713615292c427c4000cee36b86c4fe7a2). * This patch **fails SparkR unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215307046 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57211/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215302215 **[Test build #57211 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57211/consoleFull)** for PR 12683 at commit [`55523f7`](https://github.com/apache/spark/commit/55523f7713615292c427c4000cee36b86c4fe7a2). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user GayathriMurali commented on a diff in the pull request:
https://github.com/apache/spark/pull/12683#discussion_r61370053
--- Diff: R/pkg/R/mllib.R ---
@@ -406,6 +432,8 @@ ml.load <- function(path) {
jobj <- callJStatic("org.apache.spark.ml.r.RWrappers", "load", path)
if (isInstanceOf(jobj, "org.apache.spark.ml.r.NaiveBayesWrapper")) {
return(new("NaiveBayesModel", jobj = jobj))
+ } else if (isInstanceOf(jobj,
"org.apache.spark.ml.GeneralizedLinearRegressionWrapper")) {
--- End diff --
For some reason local tests(R/run-tests.sh) is not capturing these
failures. Let me fix this and submit the code
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user yanboliang commented on a diff in the pull request:
https://github.com/apache/spark/pull/12683#discussion_r61369473
--- Diff: R/pkg/R/mllib.R ---
@@ -406,6 +432,8 @@ ml.load <- function(path) {
jobj <- callJStatic("org.apache.spark.ml.r.RWrappers", "load", path)
if (isInstanceOf(jobj, "org.apache.spark.ml.r.NaiveBayesWrapper")) {
return(new("NaiveBayesModel", jobj = jobj))
+ } else if (isInstanceOf(jobj,
"org.apache.spark.ml.GeneralizedLinearRegressionWrapper")) {
--- End diff --
Should be ```org.apache.spark.ml.r.GeneralizedLinearRegressionWrapper```.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215297705 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57204/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215297703 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215297686 **[Test build #57204 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57204/consoleFull)** for PR 12683 at commit [`be0e52f`](https://github.com/apache/spark/commit/be0e52f0048cc91a04cf21e07910ee7f64dc070d). * This patch **fails SparkR unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215289906 **[Test build #57204 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57204/consoleFull)** for PR 12683 at commit [`be0e52f`](https://github.com/apache/spark/commit/be0e52f0048cc91a04cf21e07910ee7f64dc070d). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215272520 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57194/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215272519 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215272500 **[Test build #57194 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57194/consoleFull)** for PR 12683 at commit [`c044552`](https://github.com/apache/spark/commit/c044552febc582e13930fa2056ccf4d54294732f). * This patch **fails SparkR unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215266976 **[Test build #57194 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57194/consoleFull)** for PR 12683 at commit [`c044552`](https://github.com/apache/spark/commit/c044552febc582e13930fa2056ccf4d54294732f). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215260818 **[Test build #57189 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57189/consoleFull)** for PR 12683 at commit [`d2e5968`](https://github.com/apache/spark/commit/d2e5968ab889657ac1c7be57cdb2f38b52b02cb2). * This patch **fails SparkR unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215260853 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215260854 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57189/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215253529 **[Test build #57189 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57189/consoleFull)** for PR 12683 at commit [`d2e5968`](https://github.com/apache/spark/commit/d2e5968ab889657ac1c7be57cdb2f38b52b02cb2). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215234428 @yanboliang Please `LGTM` if the changes look good to you:) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215233578 @GayathriMurali Please run `dev/lint-r` locally to check R style. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215231822 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215231824 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57175/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215231813 **[Test build #57175 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57175/consoleFull)** for PR 12683 at commit [`dbacbe6`](https://github.com/apache/spark/commit/dbacbe6ed4bb9a63b0c29c09d2f225b7f1c36b1c). * This patch **fails R style tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215230891 **[Test build #57175 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57175/consoleFull)** for PR 12683 at commit [`dbacbe6`](https://github.com/apache/spark/commit/dbacbe6ed4bb9a63b0c29c09d2f225b7f1c36b1c). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215230608 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user yanboliang commented on a diff in the pull request:
https://github.com/apache/spark/pull/12683#discussion_r61067079
--- Diff: R/pkg/inst/tests/testthat/test_mllib.R ---
@@ -126,6 +126,18 @@ test_that("glm summary", {
expect_true(abs(baseSummary$deviance - 12.19313) < 1e-4)
})
+ # Test model save/load
+ modelPath <- tempfile(pattern = "GLM", fileext = ".tmp")
+ ml.save(m, modelPath)
+ expect_error(ml.save(m, modelPath))
+ ml.save(m, modelPath, overwrite = TRUE)
+ m2 <- ml.load(modelPath)
+ s2 <- summary(m2)
+ expect_equal(s$rCoefficients, s2$rCoefficients)
+ expect_equal(s$rFeatures, s2$rFeatures)
--- End diff --
I'd like to add more equality test for ```dispersion, deviance,
null.deviance, df.null, null, residual, aic```.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user yanboliang commented on a diff in the pull request:
https://github.com/apache/spark/pull/12683#discussion_r61066822
--- Diff: R/pkg/R/mllib.R ---
@@ -83,6 +83,54 @@ setMethod("glm", signature(formula = "formula", family =
"ANY", data = "SparkDat
return(new("GeneralizedLinearRegressionModel", jobj = jobj))
})
+#' Save the GeneralizedLinearRegression model to the input path.
+#'
+#' @param object A fitted GLM model
+#' @param path The directory where the model is saved
+#' @param overwrite Overwrites or not if the output path already exists.
Default is FALSE
+#' which means throw exception if the output path exists.
+#'
+#' @rdname ml.save
+#' @name ml.save
+#' @export
+#' @examples
+#' \dontrun{
+#' df <- createDataFrame(sqlContext, infert)
+#' model <- glm(education ~ ., df)
+#' path <- "path/to/model"
+#' ml.save(model, path)
+#' }
+setMethod("ml.save", signature(object =
"GeneralizedLinearRegressionModel", path = "character"),
+function(object, path, overwrite = FALSE) {
+writer <- callJMethod(object@jobj, "write")
+if (overwrite) {
+writer <- callJMethod(writer, "overwrite")
+}
+invisible(callJMethod(writer, "save", path))
+})
+
+#' Load a fitted MLlib model from the input path.
+#'
+#' @param path Path of the model to read.
+#' @return a fitted MLlib model
+#' @rdname ml.load
+#' @name ml.load
+#' @export
+#' @examples
+#' \dontrun{
+#' path <- "path/to/model"
+#' model <- ml.load(path)
+#' }
+ml.load <- function(path) {
--- End diff --
See my comments at #12680.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user yanboliang commented on a diff in the pull request:
https://github.com/apache/spark/pull/12683#discussion_r61066737
--- Diff: R/pkg/R/mllib.R ---
@@ -83,6 +83,54 @@ setMethod("glm", signature(formula = "formula", family =
"ANY", data = "SparkDat
return(new("GeneralizedLinearRegressionModel", jobj = jobj))
})
+#' Save the GeneralizedLinearRegression model to the input path.
+#'
+#' @param object A fitted GLM model
+#' @param path The directory where the model is saved
+#' @param overwrite Overwrites or not if the output path already exists.
Default is FALSE
+#' which means throw exception if the output path exists.
+#'
+#' @rdname ml.save
+#' @name ml.save
+#' @export
+#' @examples
+#' \dontrun{
+#' df <- createDataFrame(sqlContext, infert)
+#' model <- glm(education ~ ., df)
+#' path <- "path/to/model"
+#' ml.save(model, path)
+#' }
+setMethod("ml.save", signature(object =
"GeneralizedLinearRegressionModel", path = "character"),
+function(object, path, overwrite = FALSE) {
--- End diff --
indent
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-214607333 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...
GitHub user GayathriMurali opened a pull request: https://github.com/apache/spark/pull/12683 [SPARK-14315][SparkR]Add model persistence to GLMs ## What changes were proposed in this pull request? Add model persistence to GLMs in SparkR Unit tests added You can merge this pull request into a Git repository by running: $ git pull https://github.com/GayathriMurali/spark SPARK-14315 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/12683.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #12683 commit 31b2b679c041068fb60db4d36ecc28d149b04c75 Author: GayathriMurali Date: 2016-04-26T04:06:08Z [SPARK-14315][SparkR]Add model persistence to GLMs --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
