[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-08-08 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/14109 Would be great to have LOR sharing similar style and destroy mean and variance after usage. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-08-08 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/14109 LGTM. Merged into master. Great work! Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-08-06 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/14109 The current fix for broadcast variable destroy is ok. LGTM. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-08-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14109 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-08-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14109 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63288/ Test PASSed. ---

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-08-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14109 **[Test build #63288 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63288/consoleFull)** for PR 14109 at commit

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-08-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14109 **[Test build #63288 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63288/consoleFull)** for PR 14109 at commit

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-08-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14109 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63282/ Test PASSed. ---

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-08-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14109 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-08-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14109 **[Test build #63282 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63282/consoleFull)** for PR 14109 at commit

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-08-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14109 **[Test build #63282 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63282/consoleFull)** for PR 14109 at commit

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-08-05 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/14109 @dbtsai @yanboliang I went ahead and added a couple comments so someone will not mistakenly change this behavior in the future. Let me know if you see anything else, thanks! --- If your project is

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-08-04 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/14109 @sethah In my opinion, I think using `@transient lazy val` is okay since there are only two places dereferencing the `lazy val`, and we don't use it in the tight loop. LGTM except one small

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14109 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14109 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63175/ Test PASSed. ---

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14109 **[Test build #63175 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63175/consoleFull)** for PR 14109 at commit

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14109 **[Test build #63175 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63175/consoleFull)** for PR 14109 at commit

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-08-03 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/14109 @yanboliang Do you have thoughts on my comments regarding the trade-offs with using `@transient lazy val`? I am not necessarily convinced this is the best way. If it is, we should update Logistic

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-08-03 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/14109 @sethah I left two inline comments. Otherwise, LGTM. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14109 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63123/ Test PASSed. ---

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14109 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14109 **[Test build #63123 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63123/consoleFull)** for PR 14109 at commit

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14109 **[Test build #63123 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63123/consoleFull)** for PR 14109 at commit

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-08-02 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/14109 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-07-20 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/14109 also cc @mengxr - this is the same problem as in https://github.com/apache/spark/pull/13729 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14109 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62003/ Test PASSed. ---

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14109 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14109 **[Test build #62003 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62003/consoleFull)** for PR 14109 at commit

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-07-08 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/14109 ping @dbtsai I implemented this patch using `@transient` as you suggested. I ran into an obscure complication using `@transient` with `unapply` method which is not obvious. While I

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14109 **[Test build #62003 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62003/consoleFull)** for PR 14109 at commit