viirya opened a new issue, #1788: URL: https://github.com/apache/arrow-rs/issues/1788
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.** At IPC reader, we assume no validity buffer for UnionType array. But on in V4 message, only null types have no validity bitmap. We should follow C++ IPC reader to read and skip the validity buffer for V4 message. C++ IPC Reader: https://github.com/apache/arrow/blob/c715bebbd89089f385c9996560866da23ea1ddda/cpp/src/arrow/ipc/reader.cc#L262-L277 https://github.com/apache/arrow/blob/91e3ac53e2e21736ce6291d73fc37da6fa21259d/cpp/src/arrow/ipc/metadata_internal.cc#L108-L113 Currently, the Rust IPC reader cannot read V4 UnionType array. Follow the integration document, then run interop test against 0.17.1 golden file: ``` archery integration --with-cpp=true --with-rust=true --gold-dirs=/path/to/arrow/testing/data/arrow-ipc-stream/integration/0.17.1 ``` The error is: ``` RuntimeError: Command failed: /Users/liangchi/repos/arrow/rust/target/debug/arrow-json-integration-test --integration --arrow=/Users/liangchi/repos/arrow/testing/data/arrow-ipc-stream/integration/0.17.1/generated_union.arrow_file --json=/var/folders/60/w6p4p5tj64ndcfnq0wzc38580000gn/T/tmpc24ewf3_/0.17.1_union.gold.json --mode=VALIDATE With output: -------------- thread 'main' panicked at 'range end index 11 out of range for slice of length 0', library/core/src/slice/index.rs:73:5 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace ``` **Describe the solution you'd like** The interop test should be passed. **Describe alternatives you've considered** A clear and concise description of any alternative solutions or features you've considered. **Additional context** #1404 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
