GitHub user BryanCutler opened a pull request: https://github.com/apache/spark/pull/21939
[SPARK-23874][SQL][PYTHON] Upgrade Apache Arrow to 0.10.0 ## What changes were proposed in this pull request? Upgrade Apache Arrow to 0.10.0 Version 0.10.0 will allow for the following improvements and bug fixes: * Allow for adding BinaryType support * Bug fix related to array serialization ARROW-1973 * Python2 str will be made into an Arrow string instead of bytes ARROW-2101 * Python bytearrays are supported in as input to pyarrow ARROW-2141 * Java has common interface for reset to cleanup complex vectors in Spark ArrowWriter ARROW-1962 * Cleanup pyarrow type equality checks ARROW-2423 * ArrowStreamWriter should not hold references to ArrowBlocks ARROW-2632, ARROW-2645 * Improved low level handling of messages for RecordBatch ARROW-2704 ## How was this patch tested? existing tests You can merge this pull request into a Git repository by running: $ git pull https://github.com/BryanCutler/spark arrow-upgrade-010 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/21939.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #21939 ---- commit 5e4061e16947d0c7d81b86cbccfa0e222b33d48b Author: Bryan Cutler <cutlerb@...> Date: 2018-04-09T19:26:06Z made required code changes for upgrade commit 8b63e6c39e2b88ba2f4d1e9c666e7dd456d265bf Author: Bryan Cutler <cutlerb@...> Date: 2018-04-09T19:26:54Z remove unused import commit 95a1bf9de43bcee8684adf47990dc64923a63c0c Author: Bryan Cutler <cutlerb@...> Date: 2018-04-20T16:47:05Z added some TODOs commit 739a12eca9dbd83256be388703098e4d1f9cab9e Author: Bryan Cutler <cutlerb@...> Date: 2018-05-18T20:36:21Z change to using 0.10 snapshot commit d1f7355e78367a1b8a4347887fa1780106c6bffd Author: Bryan Cutler <cutlerb@...> Date: 2018-08-01T00:40:39Z replaced manifests commit abedec3243cce5fa92ea3acf711f504bab6814f3 Author: Bryan Cutler <cutlerb@...> Date: 2018-08-01T00:42:56Z removed version snapshot ---- --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org