[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[ueshin]

Github user ueshin commented on the issue:

https://github.com/apache/spark/pull/19884
  
@wesm I was able to install pyarrow 0.8.0 to my local environment via 
conda. Thanks!


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[BryanCutler]

Github user BryanCutler commented on the issue:

https://github.com/apache/spark/pull/19884
  
I just uploaded the pip packages for Windows and Linux so they are 
available.  There is an error building the Mac packages, so those will come 
later after that is resolved.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[wesm]

Github user wesm commented on the issue:

https://github.com/apache/spark/pull/19884
  
If you want to install pyarrow 0.8.0 via conda it's available now from the 
`-c conda-forge` channel (https://anaconda.org/conda-forge/pyarrow). I am not 
sure where we are at on PyPI / pip packages -- I will start the update process 
later today if no one else does cc @BryanCutler @siddharthteotia @xhochy 


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[HyukjinKwon]

Github user HyukjinKwon commented on the issue:

https://github.com/apache/spark/pull/19884
  
Just for a refreshing reminder about Jenkins, I happened to check what we 
have in Jenkins roughly a month ago (just simply by printing out the versions 
within PySpark tests) in a specific machine:

```
PyPy - No Pandas
Python 2.7 Pandas [0.16.0]
Python 3.4 Pandas [0.19.2]
```

```
PyPy - No PyArrow
python 2.7 - No PyArrow
Python 3.4 PyArrow [0.4.1]
```

I think we should also make sure which Python has the corresponding Pandas 
and PyArrow.

Also, we dropped Pandas 0.19.2 per 
http://apache-spark-developers-list.1001551.n3.nabble.com/discuss-PySpark-Can-we-drop-support-old-Pandas-lt-0-19-2-or-what-version-should-we-support-td22834.html
 and https://github.com/apache/spark/pull/19607. 
I think each Python also should have Pandas 0.19.2 now if I haven't missed 
something.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[ueshin]

Github user ueshin commented on the issue:

https://github.com/apache/spark/pull/19884
  
@wesm Yes, I'd like to use it asap to verify this patch and to confirm the 
behavior of my PR #18754 for `DecimalType` support. Thanks.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[shaneknapp]

Github user shaneknapp commented on the issue:

https://github.com/apache/spark/pull/19884
  
the install is done w/pip in a conda environment.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[wesm]

Github user wesm commented on the issue:

https://github.com/apache/spark/pull/19884
  
Just as a matter of prioritization -- do you need pip or conda packages to 
be able to proceed with finishing/verifying this patch? Getting pip packages up 
on PyPI shouldn't take too long after the release vote closes 


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/19884
  
Merged build finished. Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/19884
  
**[Test build #84743 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84743/testReport)**
 for PR 19884 at commit 
[`3a5e3c1`](https://github.com/apache/spark/commit/3a5e3c12da0bd8cdcb580067281b908422f35a6d).
 * This patch **fails Scala style tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/19884
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84743/
Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/19884
  
**[Test build #84743 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84743/testReport)**
 for PR 19884 at commit 
[`3a5e3c1`](https://github.com/apache/spark/commit/3a5e3c12da0bd8cdcb580067281b908422f35a6d).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[BryanCutler]

Github user BryanCutler commented on the issue:

https://github.com/apache/spark/pull/19884
  
>yeah, i can do the upgrade next week. i'll be working remotely from the 
east coast, but unavailable at all on monday due to travel.

Great, thanks @shaneknapp !  I'll ping you when I think we are set to go


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/19884
  
Merged build finished. Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/19884
  
**[Test build #84740 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84740/testReport)**
 for PR 19884 at commit 
[`c3d612f`](https://github.com/apache/spark/commit/c3d612fd1151271d4da027e3d252eed0aea4ea60).
 * This patch **fails Scala style tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/19884
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84740/
Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/19884
  
**[Test build #84740 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84740/testReport)**
 for PR 19884 at commit 
[`c3d612f`](https://github.com/apache/spark/commit/c3d612fd1151271d4da027e3d252eed0aea4ea60).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/19884
  
Merged build finished. Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/19884
  
**[Test build #84738 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84738/testReport)**
 for PR 19884 at commit 
[`46ad595`](https://github.com/apache/spark/commit/46ad5951652c40de3c2c108c9b952b16dfcc3ad5).
 * This patch **fails Scala style tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/19884
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84738/
Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/19884
  
**[Test build #84738 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84738/testReport)**
 for PR 19884 at commit 
[`46ad595`](https://github.com/apache/spark/commit/46ad5951652c40de3c2c108c9b952b16dfcc3ad5).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[wesm]

Github user wesm commented on the issue:

https://github.com/apache/spark/pull/19884
  
The Arrow 0.8.0 release vote just started today. Assuming it passes, the 
earliest you could see packages pushed to PyPI or conda-forge would be sometime 
on Thursday evening or Friday. 


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[shaneknapp]

Github user shaneknapp commented on the issue:

https://github.com/apache/spark/pull/19884
  
yeah, i can do the upgrade next week.  i'll be working remotely from the
east coast, but unavailable at all on monday due to travel.

On Mon, Dec 11, 2017 at 1:59 PM, Bryan Cutler 
wrote:

> Jenkins cannot support to install multiple versions of PyArrow?
>
> @zsxwing  that's right, we will have to
> coordinate to make sure the Jenkins pyarrow is upgraded to version 0.8 as
> well. I'm not sure the best way to coordinate all of this because this PR,
> jenkins upgrade, and Spark Netty upgrade all need to happen at the same
> time.
>
> @holdenk  @shaneknapp
>  will one of you be able to work on the
> pyarrow upgrade for Jenkins sometime around next week? (assuming Arrow 0.8
> is released in the next day or so)
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> , or 
mute
> the thread
> 

> .
>



---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[zsxwing]

Github user zsxwing commented on the issue:

https://github.com/apache/spark/pull/19884
  
@BryanCutler could you just pull my changes into this PR since we need to 
both changes to pass Jenkins? Thanks!


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[BryanCutler]

Github user BryanCutler commented on the issue:

https://github.com/apache/spark/pull/19884
  
>Jenkins cannot support to install multiple versions of PyArrow?

@zsxwing that's right, we will have to coordinate to make sure the Jenkins 
pyarrow is upgraded to version 0.8 as well.  I'm not sure the best way to 
coordinate all of this because this PR, jenkins upgrade, and Spark Netty 
upgrade all need to happen at the same time.

@holdenk @shaneknapp will one of you be able to work on the pyarrow upgrade 
for Jenkins sometime around next week?  (assuming Arrow 0.8 is released in the 
next day or so)


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[BryanCutler]

Github user BryanCutler commented on the issue:

https://github.com/apache/spark/pull/19884
  
>When I tried to run tests locally, I got OutOfMemoryException

@ueshin , you got that error because the latest Arrow has upgraded Netty to 
4.1.17 but Spark has an older version on the classpath.  If you apply #19829 on 
top of this PR, the tests should pass.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[zsxwing]

Github user zsxwing commented on the issue:

https://github.com/apache/spark/pull/19884
  
I saw #18974 tried to upgrade Arrow but got closed due to some Jenkins 
issue. @ueshin do you have any idea what may block this PR? Jenkins cannot 
support to install multiple versions of PyArrow?


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/19884
  
**[Test build #84663 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84663/testReport)**
 for PR 19884 at commit 
[`fdba406`](https://github.com/apache/spark/commit/fdba406f29216b8ef592de45dc36461217113410).
 * This patch **fails Scala style tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/19884
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84663/
Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/19884
  
Merged build finished. Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/19884
  
**[Test build #84663 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84663/testReport)**
 for PR 19884 at commit 
[`fdba406`](https://github.com/apache/spark/commit/fdba406f29216b8ef592de45dc36461217113410).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/19884
  
**[Test build #84616 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84616/testReport)**
 for PR 19884 at commit 
[`93b1eb3`](https://github.com/apache/spark/commit/93b1eb37fa1f39d5d69853248f72808bf3b05a81).
 * This patch **fails Scala style tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/19884
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84616/
Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/19884
  
Merged build finished. Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/19884
  
**[Test build #84616 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84616/testReport)**
 for PR 19884 at commit 
[`93b1eb3`](https://github.com/apache/spark/commit/93b1eb37fa1f39d5d69853248f72808bf3b05a81).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[BryanCutler]

Github user BryanCutler commented on the issue:

https://github.com/apache/spark/pull/19884
  
@zsxwing, fyi after applying your Netty upgrade patch to Arrow, and then 
your other patch for Spark, all of the Spark Scala/Java tests pass


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[HyukjinKwon]

Github user HyukjinKwon commented on the issue:

https://github.com/apache/spark/pull/19884
  
cc @zsxwing as well, I saw you opened a JIRA about this - SPARK-22656



---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[BryanCutler]

Github user BryanCutler commented on the issue:

https://github.com/apache/spark/pull/19884
  
> Great, @BryanCutler . Could you put the highlight in the PR description, 
too?

Sure, thanks @dongjoon-hyun !  Will do, just want to go back and check the 
release notes first


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[BryanCutler]

Github user BryanCutler commented on the issue:

https://github.com/apache/spark/pull/19884
  
This is a WIP to start updating Spark to use Arrow 0.8.0 which will be 
released soon.

TODO:

- [ ] Update to reflect Java API changes
- [ ] Update to reflect Python API changes
- [ ] Use new Python type checking
- [ ] Remove Python type casting workarounds


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[dongjoon-hyun]

Github user dongjoon-hyun commented on the issue:

https://github.com/apache/spark/pull/19884
  
Great, @BryanCutler . Could you put the highlight in the PR description, 
too?


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[BryanCutler]

Github user BryanCutler commented on the issue:

https://github.com/apache/spark/pull/19884
  
The highlights that pertain to Spark for the update from Arrow versoin 
0.4.1 to 0.8.0 include:

* Java refactoring for more simple API
* Type support for DecimalType, ArrayType
* Improved type casting support in Python
* 


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/19884
  
**[Test build #84447 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84447/testReport)**
 for PR 19884 at commit 
[`4b0790b`](https://github.com/apache/spark/commit/4b0790bfdd281719c2b5471c077e495523b28e3b).
 * This patch **fails Scala style tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/19884
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84447/
Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/19884
  
Merged build finished. Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/19884
  
**[Test build #84447 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84447/testReport)**
 for PR 19884 at commit 
[`4b0790b`](https://github.com/apache/spark/commit/4b0790bfdd281719c2b5471c077e495523b28e3b).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org