[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/14109 Would be great to have LOR sharing similar style and destroy mean and variance after usage. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/14109 LGTM. Merged into master. Great work! Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/14109 The current fix for broadcast variable destroy is ok. LGTM. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14109 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14109 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63288/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14109 **[Test build #63288 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63288/consoleFull)** for PR 14109 at commit [`9c2bf47`](https://github.com/apache/spark/commit/9c2bf479ea8a82ee893716416bd364d3b1324c20). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14109 **[Test build #63288 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63288/consoleFull)** for PR 14109 at commit [`9c2bf47`](https://github.com/apache/spark/commit/9c2bf479ea8a82ee893716416bd364d3b1324c20). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14109 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63282/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14109 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14109 **[Test build #63282 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63282/consoleFull)** for PR 14109 at commit [`0d99795`](https://github.com/apache/spark/commit/0d9979582d9ed2c9869452f5a0d44d46ebaf9aeb). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14109 **[Test build #63282 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63282/consoleFull)** for PR 14109 at commit [`0d99795`](https://github.com/apache/spark/commit/0d9979582d9ed2c9869452f5a0d44d46ebaf9aeb). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user sethah commented on the issue: https://github.com/apache/spark/pull/14109 @dbtsai @yanboliang I went ahead and added a couple comments so someone will not mistakenly change this behavior in the future. Let me know if you see anything else, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/14109 @sethah In my opinion, I think using `@transient lazy val` is okay since there are only two places dereferencing the `lazy val`, and we don't use it in the tight loop. LGTM except one small comment. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14109 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14109 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63175/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14109 **[Test build #63175 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63175/consoleFull)** for PR 14109 at commit [`10ba14e`](https://github.com/apache/spark/commit/10ba14e1bd56e7e51513c8399b75116e54add023). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14109 **[Test build #63175 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63175/consoleFull)** for PR 14109 at commit [`10ba14e`](https://github.com/apache/spark/commit/10ba14e1bd56e7e51513c8399b75116e54add023). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user sethah commented on the issue: https://github.com/apache/spark/pull/14109 @yanboliang Do you have thoughts on my comments regarding the trade-offs with using `@transient lazy val`? I am not necessarily convinced this is the best way. If it is, we should update Logistic Regression to use this method as well. I'll address your other comments shortly. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/14109 @sethah I left two inline comments. Otherwise, LGTM. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14109 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63123/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14109 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14109 **[Test build #63123 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63123/consoleFull)** for PR 14109 at commit [`53c9192`](https://github.com/apache/spark/commit/53c9192f0a8c234a1d185e3066154861ef25b77f). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14109 **[Test build #63123 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63123/consoleFull)** for PR 14109 at commit [`53c9192`](https://github.com/apache/spark/commit/53c9192f0a8c234a1d185e3066154861ef25b77f). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/14109 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user sethah commented on the issue: https://github.com/apache/spark/pull/14109 also cc @mengxr - this is the same problem as in https://github.com/apache/spark/pull/13729 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14109 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62003/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14109 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14109 **[Test build #62003 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62003/consoleFull)** for PR 14109 at commit [`53c9192`](https://github.com/apache/spark/commit/53c9192f0a8c234a1d185e3066154861ef25b77f). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user sethah commented on the issue: https://github.com/apache/spark/pull/14109 ping @dbtsai I implemented this patch using `@transient` as you suggested. I ran into an obscure complication using `@transient` with `unapply` method which is not obvious. While I understand that it is nice to keep the `add` method signature unchanged, I think you can argue that it is more appropriate to pass the coefficients and featuresStd arrays directly to the method since it is the only place in the class they are used. Add that to the fact that the `@transient` approach has a more confusing implementation and could potentially be unknowningly undone by future developers, it may not be the best approach. I am open to feedback/suggestions. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14109 **[Test build #62003 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62003/consoleFull)** for PR 14109 at commit [`53c9192`](https://github.com/apache/spark/commit/53c9192f0a8c234a1d185e3066154861ef25b77f). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
