[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2018-01-02 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20114 Thanks @HyukjinKwon and @ueshin ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2018-01-01 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20114 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20114 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85554/ Test PASSed. ---

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20114 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20114 **[Test build #85554 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85554/testReport)** for PR 20114 at commit

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20114 **[Test build #85554 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85554/testReport)** for PR 20114 at commit

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-31 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20114 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20114 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85552/ Test FAILed. ---

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20114 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20114 **[Test build #85552 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85552/testReport)** for PR 20114 at commit

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20114 **[Test build #85552 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85552/testReport)** for PR 20114 at commit

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20114 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20114 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85533/ Test PASSed. ---

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20114 **[Test build #85533 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85533/testReport)** for PR 20114 at commit

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-29 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20114 The new workaround seems to be fine and I also added another test with array null values to test that along with all non-null values. ---

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20114 **[Test build #85533 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85533/testReport)** for PR 20114 at commit

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-29 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20114 > How about simply returning false from ArrowVectorAccessor.isNullAt(int rowId) when accessor.getValueCount() > 0 && accessor.getValidityBuffer().capacity() == 0 Good idea @ueshin , I

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20114 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20114 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85505/ Test PASSed. ---

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20114 **[Test build #85505 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85505/testReport)** for PR 20114 at commit

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20114 **[Test build #85505 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85505/testReport)** for PR 20114 at commit

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-29 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20114 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20114 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20114 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85499/ Test FAILed. ---

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20114 **[Test build #85499 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85499/testReport)** for PR 20114 at commit

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-28 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20114 How about simply returning `false` from `ArrowVectorAccessor.isNullAt(int rowId)` when `accessor.getValueCount() > 0 && accessor.getValidityBuffer().capacity() == 0` without modifying the buffer?

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-28 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20114 ping @ueshin @HyukjinKwon Unfortunately, there was a bug in the Arrow 0.8.0 release on the Java side https://issues.apache.org/jira/browse/ARROW-1948 that caused a problem here. I was

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20114 **[Test build #85499 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85499/testReport)** for PR 20114 at commit