pchintar opened a new pull request, #9822:
URL: https://github.com/apache/arrow-rs/pull/9822
# Which issue does this PR close?
- Closes #9821 .
# Rationale for this change
`skip_field` currently uses a wildcard match (`_`) to skip remaining
`DataType` variants by assuming a two-buffer layout. This is not robust, as new
variants may have different buffer layouts, leading to incorrect skipping and
buffer misalignment.
Replacing the wildcard with explicit handling ensures the correct number of
buffers are skipped and avoids silent errors for future types.
# What changes are included in this PR?
* Removed the wildcard (`_`) match arm in `skip_field`
* Added explicit handling for all remaining fixed-width and boolean
`DataType` variants
* Each of these types now explicitly skips:
* null buffer
* values buffer
File updated:
* `arrow-ipc/src/reader.rs`
# Are these changes tested?
Yes.
* Added test: `test_projection_skip_fixed_width_types` in
`arrow-ipc/src/reader.rs`
* The test iterates over all fixed-width and boolean `DataType` variants
covered by this change
* For each type:
* writes a batch with `[skipped_column(type), values_column(Int32)]`
* reads only the second column (skipping the first)
* verifies whether the returned column exactly matches the original
`Int32` values
* This directly validates that skipping each of these types consumes the
correct number of buffers
* Also, all existing `arrow-ipc` tests pass (`cargo test -p arrow-ipc --lib`)
# Are there any user-facing changes?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
