[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19349 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19349 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82231/ Test PASSed. ---

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19349 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19349 **[Test build #82231 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82231/testReport)** for PR 19349 at commit

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19349 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82223/ Test PASSed. ---

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19349 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19349 **[Test build #82223 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82223/testReport)** for PR 19349 at commit

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19349 **[Test build #82231 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82231/testReport)** for PR 19349 at commit

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19349 LGTM too. Should be good to go. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19349 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19349 **[Test build #82221 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82221/testReport)** for PR 19349 at commit

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19349 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82221/ Test FAILed. ---

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19349 **[Test build #82223 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82223/testReport)** for PR 19349 at commit

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-27 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19349 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19349 **[Test build #82221 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82221/testReport)** for PR 19349 at commit

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19349 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82214/ Test PASSed. ---

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19349 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19349 **[Test build #82214 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82214/testReport)** for PR 19349 at commit

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19349 **[Test build #82214 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82214/testReport)** for PR 19349 at commit

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/19349 Nice job on refactoring `PythonRunner`! I think we should just replace the arrow file format with stream format for pandas udf instead of having a new conf to enable it, as long as all the

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19349 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19349 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19349 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82184/ Test PASSed. ---

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19349 **[Test build #82184 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82184/testReport)** for PR 19349 at commit

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19349 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82183/ Test PASSed. ---

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19349 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19349 **[Test build #82183 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82183/testReport)** for PR 19349 at commit

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19349 **[Test build #82184 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82184/testReport)** for PR 19349 at commit

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19349 **[Test build #82183 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82183/testReport)** for PR 19349 at commit

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19349 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19349 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82180/ Test FAILed. ---

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19349 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19349 **[Test build #82180 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82180/testReport)** for PR 19349 at commit

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/19349 The performance test I did in my local based on @BryanCutler's (https://github.com/apache/spark/pull/18659#issuecomment-315879173) is as follows: ```python from pyspark.sql.functions

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19349 **[Test build #82180 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82180/testReport)** for PR 19349 at commit

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19349 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19349 **[Test build #82175 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82175/testReport)** for PR 19349 at commit

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19349 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82175/ Test FAILed. ---

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19349 **[Test build #82175 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82175/testReport)** for PR 19349 at commit

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-25 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/19349 cc @BryanCutler @HyukjinKwon @viirya @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For