haohuaijin opened a new issue, #10425:
URL: https://github.com/apache/arrow-rs/issues/10425
**Is your feature request related to a problem or challenge? Please describe
what you are trying to do.**
Follow up to #10141.
When the two bitmaps have different lengths, `intersect_masks` and
`union_masks`
(`parquet/src/arrow/arrow_reader/selection/boolean.rs`) currently:
1. slice both inputs to the common prefix,
2. compute `&` / `|` over the prefix (one allocation),
3. append the prefix and the longer input's tail into a
`BooleanBufferBuilder` (a second allocation plus two copies).
As suggested by @alamb in
https://github.com/apache/arrow-rs/pull/10141#discussion_r3641586163, this
can
be done with a single allocation: copy the longer buffer once, then apply
`&=` / `|=` over the common prefix in place.
**Describe the solution you'd like**
Copy the longer buffer into a `MutableBuffer` and combine the shorter one
into
its prefix in place (e.g. via the bitwise helpers in `arrow_buffer::buffer`),
avoiding the intermediate prefix buffer and the builder append copies. Care
is
needed for non-byte-aligned cases: the inputs may have a non-zero bit offset
(e.g. produced by `BooleanBuffer::slice`), and the split point at the common
prefix is not generally byte-aligned.
**Describe alternatives you've considered**
Keep the current implementation — it is correct and simple, this is purely an
allocation/copy optimization.
**Additional context**
The tail-passthrough semantics (bits beyond the common length are taken from
the longer input unchanged, for both intersect and union) are covered by
existing unit tests in `selection/boolean.rs`; those tests should keep
passing
unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
