[GitHub] spark issue #21013: [WIP][SPARK-23874][SQL][PYTHON] Upgrade Arrow and pyarro...

2018-04-10 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21013 > do you suggest we should test with the minimum version and current version of pyarrow? yup. --- - To

[GitHub] spark issue #21013: [WIP][SPARK-23874][SQL][PYTHON] Upgrade Arrow and pyarro...

2018-04-10 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/21013 Closing this now, will reopen when the issue is resolved in the next Arrow release --- - To unsubscribe, e-mail:

[GitHub] spark issue #21013: [WIP][SPARK-23874][SQL][PYTHON] Upgrade Arrow and pyarro...

2018-04-10 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/21013 Yeah, we should be able to keep the minimum version the same, afaik there is no break in compatibility. @HyukjinKwon so do you suggest we should test with the minimum version and current

[GitHub] spark issue #21013: [WIP][SPARK-23874][SQL][PYTHON] Upgrade Arrow and pyarro...

2018-04-10 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21013 I think it should work for 0.8.0+ too in perspective of Python and I think it should be best to have at least minimum combination for those versions ... Wait wait .. this PR doesn't

[GitHub] spark issue #21013: [WIP][SPARK-23874][SQL][PYTHON] Upgrade Arrow and pyarro...

2018-04-10 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21013 Yea ... I think that's actually the most tricky one in this issue. We have a similar issue about it in R IIRC. --- - To

[GitHub] spark issue #21013: [WIP][SPARK-23874][SQL][PYTHON] Upgrade Arrow and pyarro...

2018-04-10 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21013 hmm, btw, if we upgrade arrow on Jenkins, how does it affect other branches like 2.3? --- - To unsubscribe, e-mail:

[GitHub] spark issue #21013: [WIP][SPARK-23874][SQL][PYTHON] Upgrade Arrow and pyarro...

2018-04-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21013 @BryanCutler Yeah, we are unable to upgrade to 0.9 until the regression is fixed. Thanks for working on it! --- - To

[GitHub] spark issue #21013: [WIP][SPARK-23874][SQL][PYTHON] Upgrade Arrow and pyarro...

2018-04-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21013 cc @ueshin @yhuai --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21013: [WIP][SPARK-23874][SQL][PYTHON] Upgrade Arrow and pyarro...

2018-04-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21013 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21013: [WIP][SPARK-23874][SQL][PYTHON] Upgrade Arrow and pyarro...

2018-04-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21013 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89075/ Test FAILed. ---

[GitHub] spark issue #21013: [WIP][SPARK-23874][SQL][PYTHON] Upgrade Arrow and pyarro...

2018-04-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21013 **[Test build #89075 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89075/testReport)** for PR 21013 at commit

[GitHub] spark issue #21013: [WIP][SPARK-23874][SQL][PYTHON] Upgrade Arrow and pyarro...

2018-04-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21013 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21013: [WIP][SPARK-23874][SQL][PYTHON] Upgrade Arrow and pyarro...

2018-04-09 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/21013 The code changes are pretty much done, but I hit a regression in Python regarding conversion of `DecimalType` with `None` values. I filed to https://issues.apache.org/jira/browse/ARROW-2432 to

[GitHub] spark issue #21013: [WIP][SPARK-23874][SQL][PYTHON] Upgrade Arrow and pyarro...

2018-04-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21013 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2113/

[GitHub] spark issue #21013: [WIP][SPARK-23874][SQL][PYTHON] Upgrade Arrow and pyarro...

2018-04-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21013 **[Test build #89075 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89075/testReport)** for PR 21013 at commit