[GitHub] spark issue #13729: [SPARK-16008][ML] Remove unnecessary serialization in lo...
Github user sethah commented on the issue: https://github.com/apache/spark/pull/13729 @dbtsai I'll take a look later this week --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13729: [SPARK-16008][ML] Remove unnecessary serialization in lo...
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/13729 Hi @jodersky @sethah Could you test in Linear Regression, if `@transient` helps the performance for the same serialization issue? https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13729: [SPARK-16008][ML] Remove unnecessary serialization in lo...
Github user jodersky commented on the issue: https://github.com/apache/spark/pull/13729 Hi @dbtsai, I assisted @sethah with some serialization issues during this PR. I know we considered using transient but can't recall exactly why we ended up not. My knowledge about the bigger picture of this PR is quite limited, but one explanation that comes to mind is that the `coefficients` and `featuresStd` parameters are only used within the `add` method. So the reasoning was to keep parameters as local as possible. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13729: [SPARK-16008][ML] Remove unnecessary serialization in lo...
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/13729 @sethah Late comment. Great improvement for high dimensional problems. I didn't test it out myself, and I wonder whether `@transient` annotation works in the constructor of `LogisticAggregator`. Thus, the code will be cleaner with using `c.add(instance)`. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13729: [SPARK-16008][ML] Remove unnecessary serialization in lo...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13729 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13729: [SPARK-16008][ML] Remove unnecessary serialization in lo...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13729 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60712/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13729: [SPARK-16008][ML] Remove unnecessary serialization in lo...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13729 **[Test build #60712 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60712/consoleFull)** for PR 13729 at commit [`5d668a6`](https://github.com/apache/spark/commit/5d668a6f93859801262393540fe954257f433a35). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13729: [SPARK-16008][ML] Remove unnecessary serialization in lo...
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/13729 Nice catch and LGTM! Merging into master and branch-2.0. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13729: [SPARK-16008][ML] Remove unnecessary serialization in lo...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13729 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13729: [SPARK-16008][ML] Remove unnecessary serialization in lo...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13729 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60710/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13729: [SPARK-16008][ML] Remove unnecessary serialization in lo...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13729 **[Test build #60712 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60712/consoleFull)** for PR 13729 at commit [`5d668a6`](https://github.com/apache/spark/commit/5d668a6f93859801262393540fe954257f433a35). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13729: [SPARK-16008][ML] Remove unnecessary serialization in lo...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13729 **[Test build #60710 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60710/consoleFull)** for PR 13729 at commit [`96b0a45`](https://github.com/apache/spark/commit/96b0a4505b4a43bc254065e084fb9b72b1e4a92b). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13729: [SPARK-16008][ML] Remove unnecessary serialization in lo...
Github user sethah commented on the issue: https://github.com/apache/spark/pull/13729 @srowen Thanks for the review! I responded to your comments, let me know what you think. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13729: [SPARK-16008][ML] Remove unnecessary serialization in lo...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13729 **[Test build #60710 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60710/consoleFull)** for PR 13729 at commit [`96b0a45`](https://github.com/apache/spark/commit/96b0a4505b4a43bc254065e084fb9b72b1e4a92b). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13729: [SPARK-16008][ML] Remove unnecessary serialization in lo...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/13729 I think that makes sense. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13729: [SPARK-16008][ML] Remove unnecessary serialization in lo...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13729 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60681/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13729: [SPARK-16008][ML] Remove unnecessary serialization in lo...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13729 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13729: [SPARK-16008][ML] Remove unnecessary serialization in lo...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13729 **[Test build #60681 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60681/consoleFull)** for PR 13729 at commit [`ef8fdea`](https://github.com/apache/spark/commit/ef8fdea808052846055979c642b5f47255ee9e3d). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13729: [SPARK-16008][ML] Remove unnecessary serialization in lo...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13729 **[Test build #60681 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60681/consoleFull)** for PR 13729 at commit [`ef8fdea`](https://github.com/apache/spark/commit/ef8fdea808052846055979c642b5f47255ee9e3d). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13729: [SPARK-16008][ML] Remove unnecessary serialization in lo...
Github user sethah commented on the issue: https://github.com/apache/spark/pull/13729 cc @jkbradley @dbtsai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
