viirya opened a new pull request, #11362: URL: https://github.com/apache/arrow-rs/pull/11362
# Which issue does this PR close? No linked issue. # Rationale for this change FixedSizeList nullability validation compares expanded parent validity with the unsliced child validity, ignoring the parent offset. This can both reject valid arrays and accept nulls in non-nullable items. For example, with list size 2, parent offset 1 and length 2, and child values `[0, 1, null, null, 4, 5]`, parent validity `[false, true]` should be accepted but is rejected; `[true, false]` should be rejected but is accepted. When the parent has no null buffer, child nulls outside the visible range can also cause an incorrect rejection. # What changes are included in this PR? - Slice child validity to `offset * list_size .. (offset + len) * list_size` before checking non-nullable items. - Include the parent offset in the required child length check so validation rejects undersized children before slicing. - Add regression coverage for masked and visible child nulls, child offsets, absent parent null buffers, nulls outside the visible range, empty slices, zero-sized lists, and undersized children. # Are these changes tested? All three new regression tests failed before the fix and pass afterward. - `cargo test -p arrow-buffer -p arrow-data -p arrow-array -p arrow-select -p arrow-ipc --lib`: 1,690 passed, 1 ignored. - `cargo clippy -p arrow-data --all-targets -- -D warnings`: passed. - `cargo fmt --all -- --check`: passed. - `git diff --check`: passed. # Are there any user-facing changes? Validation now correctly accepts child nulls masked by a sliced FixedSizeList parent or outside its visible range, and rejects visible nulls in non-nullable items. Undersized child arrays are rejected when the parent offset requires additional elements. No public API changes. # AI assistance AI assistance was used to investigate the validation semantics, implement the fix and regression tests, review the diff, run the checks above, and prepare this PR description. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
