[
https://issues.apache.org/jira/browse/SPARK-33213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17219840#comment-17219840
]
Bryan Cutler commented on SPARK-33213:
--------------------------------------
Just a couple notes:
The library and format versions are now split, the format version is still at
1.0.0 so remains binary compatible, see here for more info
[https://arrow.apache.org/blog/2020/10/22/2.0.0-release/]
I don't think there are any relevant changes in Arrow Java between 1.0.1 and
2.0.0, and pyspark is currently working with pyarrow 2.0.0 with the added env
var {{PYARROW_IGNORE_TIMEZONE=1}}
> Upgrade Apache Arrow to 2.0.0
> -----------------------------
>
> Key: SPARK-33213
> URL: https://issues.apache.org/jira/browse/SPARK-33213
> Project: Spark
> Issue Type: Dependency upgrade
> Components: SQL
> Affects Versions: 3.0.1
> Reporter: Chao Sun
> Priority: Minor
>
> Apache Arrow 2.0.0 has [just been
> released|https://cwiki.apache.org/confluence/display/ARROW/Arrow+2.0.0+Release].
> This proposes to upgrade Spark's Arrow dependency to use 2.0.0, from the
> current 1.0.1.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
