[
https://issues.apache.org/jira/browse/SPARK-32312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17190553#comment-17190553
]
Bryan Cutler commented on SPARK-32312:
--------------------------------------
Sorry for the delay, I was holding off for a couple of things. Version 1.0.1
was being released and wanted to see if there was anything crucial to use that
version instead. Also, to see if ARROW-9528 would be included in the 1.0.1
release, which would have broken our Python tests, but it will be in v2.0.0.
For Arrow Java, there were no real fixes between 1.0.0 and 1.0.1, so it won't
really make a difference, but might as well use 1.0.1. For pyarrow, there were
no fixes that affect our Spark usage, so it should be safe to use a minimum
version of 1.0.0. I will try to put up a PR tomorrow.
> Upgrade Apache Arrow to 1.0.0
> -----------------------------
>
> Key: SPARK-32312
> URL: https://issues.apache.org/jira/browse/SPARK-32312
> Project: Spark
> Issue Type: Improvement
> Components: PySpark, SQL
> Affects Versions: 3.0.0
> Reporter: Bryan Cutler
> Priority: Major
>
> Apache Arrow will soon release v1.0.0 which provides backward/forward
> compatibility guarantees as well as a number of fixes and improvements. This
> will upgrade the Java artifact and PySpark API. Although PySpark will not
> need special changes, it might be a good idea to bump up minimum supported
> version and CI testing.
> TBD: list of important improvements and fixes
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
