yongster opened a new pull request, #10994:
URL: https://github.com/apache/arrow-rs/pull/10994
`take` can introduce output nulls when its indices contain nulls.
Most Arrow arrays represent these with a top-level validity bitmap. However,
RunEndEncoded and Union arrays derive logical nullability from their child
arrays:
- A RunEndEncoded array derives nullability from its `values` field.
- A Dense Union represents a null through a nullable selected child.
- A Sparse Union stores every child at every output position.
Previously, `take` could write nulls into children whose corresponding field
metadata was marked as non-nullable. This made the physical output
inconsistent
with its declared schema.
- Closes #10992.
- Return a compute error when null take indices would introduce nulls into a
RunEndEncoded array with a non-nullable `values` field.
- For Dense Union arrays, select a nullable child to represent null take
indices.
- Return a compute error for Sparse Union arrays with non-nullable fields
when
take indices contain nulls, since every child would otherwise receive an
introduced null.
- Add regression tests for each behavior.
This complements #10909 and ensures that arrays marked as non-nullable do not
produce output containing newly introduced nulls.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
