Jefffrey commented on issue #18760: URL: https://github.com/apache/arrow/issues/18760#issuecomment-5193679004
Stumbled upon this whilst refreshing my knowledge of union arrays. Some reference reading too: - https://lists.apache.org/thread/8pgv7tsko4mlxjzptor0ool98j6m578c - https://lists.apache.org/thread/cj5dh060kdrz2vdhgrwzmmkyw986nyc7 Did some quick research on how the implementations are handling validation of this (with some help from chatgpt to identify relevant code locations; I'm not as familiar with implementations other than Rust so feel free to correct me if anything is incorrect here) - Not accounting for how they may build the union arrays ## Rust/Java/nanoarrow These implementations only check offsets are within bounds, not enforcing them to be increasing: - https://github.com/apache/arrow-rs/blob/7e4432e997e5b0be31a5641e078a889d82b6f61b/arrow-array/src/array/union_array.rs#L230 - https://github.com/apache/arrow-java/blob/06170242bde2f492e068235efdd2183a3cbd87d7/vector/src/main/java/org/apache/arrow/vector/validate/ValidateVectorDataVisitor.java#L185 - https://github.com/apache/arrow-nanoarrow/blob/a59c5a0e18d607a6a43d3c05bda192c6480d3a62/src/nanoarrow/common/array.c#L1604 ## C++/Go Validates according to the spec that offsets are monotonic, though doesn't enforce them to be strictly increasing. - https://github.com/apache/arrow/blob/906bc0aea74051d857bc9e676fb53522d3e09fd3/cpp/src/arrow/array/validate.cc#L390-L408 - https://github.com/apache/arrow-go/blob/666f8dc2ea470d69fc294ba1f850d04050fb5047/arrow/array/union.go#L209 ## .NET/JavaScript/Julia/Swift Couldn't find/not present -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
