GitHub user BryanCutler opened a pull request:
https://github.com/apache/spark/pull/19284
[SPARK-22067][SQL] ArrowWriter should use position when setting UTF8String
ByteBuffer
## What changes were proposed in this pull request?
The ArrowWriter StringWriter was setting Arrow data using a position of 0
instead of the actual position in the ByteBuffer. This was currently working
because of a bug ARROW-1447, and has been fixed as of
Arrow 0.7.0. Testing with this version revealed the error in
ArrowConvertersSuite test string conversion.
## How was this patch tested?
Existing tests, manually verified working with Arrow 0.7.0
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/BryanCutler/spark
arrow-ArrowWriter-StringWriter-position-SPARK-22067
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/19284.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #19284
----
commit 5ac572d3cb2422f57e101fa2cbc761f4b748daa6
Author: Bryan Cutler <cutl...@gmail.com>
Date: 2017-09-19T22:17:23Z
pass in position of ByteBuffer when using StringWriter
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
