Fokko opened a new pull request, #902:
URL: https://github.com/apache/iceberg-python/pull/902
PyIceberg struggled with different types of arrow, such as the `string` and
`large_string`. They represent the same but are different under the hood.
My take is that we should hide this kind of detail from the user as much as
possible. Now we went down the road of passing in the Iceberg schema into
Arrow, but when doing this, Iceberg has to decide if it is a large or non-large
type.
This PR removes passing down the schema to let Arrow decide unless:
- The type should be evolved
- In case of re-ordering, we reorder the original types
Let me know what you think @syun64
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
