[GitHub] spark pull request: [SPARK-14031] [SQL] speedup CSV writer

2016-05-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13229 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-14031] [SQL] speedup CSV writer

2016-05-23 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/13229#issuecomment-221043731 LGTM, merging to master and branch-2.0. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-14031] [SQL] speedup CSV writer

2016-05-20 Thread lresende
Github user lresende commented on the pull request: https://github.com/apache/spark/pull/13229#issuecomment-220753618 LGTM, I also verified with a issue discussed on the mailing list where writing to csv was taking about 1hr, and with this change it only took a few minutes. --- If

[GitHub] spark pull request: [SPARK-14031] [SQL] speedup CSV writer

2016-05-20 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/13229#issuecomment-220732619 cc @liancheng @falaki --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-14031] [SQL] speedup CSV writer

2016-05-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13229#issuecomment-220728625 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-14031] [SQL] speedup CSV writer

2016-05-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13229#issuecomment-220728629 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-14031] [SQL] speedup CSV writer

2016-05-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13229#issuecomment-220728423 **[Test build #59023 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59023/consoleFull)** for PR 13229 at commit

[GitHub] spark pull request: [SPARK-14031] [SQL] speedup CSV writer

2016-05-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13229#issuecomment-220710358 **[Test build #59023 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59023/consoleFull)** for PR 13229 at commit

[GitHub] spark pull request: [SPARK-14031] [SQL] speedup CSV writer

2016-05-20 Thread davies
GitHub user davies opened a pull request: https://github.com/apache/spark/pull/13229 [SPARK-14031] [SQL] speedup CSV writer ## What changes were proposed in this pull request? Currently, we create an CSVWriter for every row, it's very expensive and memory hungry, took