Jefffrey commented on PR #10438:
URL: https://github.com/apache/arrow-rs/pull/10438#issuecomment-5308321729
how about something like this
```rust
let mut_buffer_result = self.values.into_inner().into_mutable();
match mut_buffer_result {
Ok(mut mutable_buffer) => {
let raw_bytes = mutable_buffer.as_slice_mut();
let byte_idx_of_end = end / 8;
let bits_to_preserve = end % 8;
// end on a byte boundary, so just easily rewrite at byte
level
if bits_to_preserve == 0 {
// TODO: this technically can modify bits beyond what the
// boolean buffer actually points to, but given we
// have unique ownership it should be fine? there
could
// be pathological case where if this buffer was
sliced
// there could be unused bytes that we still
process,
// if we wanna bother with that edge case
raw_bytes
.iter_mut()
.skip(byte_idx_of_end)
.for_each(|b| *b = 0);
} else {
// if end in middle of a byte, need to unset only higher
bits
raw_bytes[byte_idx_of_end] &= (1_u8 << bits_to_preserve)
- 1;
raw_bytes
.iter_mut()
// +1 since we account for one byte above
.skip(byte_idx_of_end + 1)
.for_each(|b| *b = 0);
}
// TODO: this offset is wrong?
let boolean_buf = BooleanBuffer::new(mutable_buffer.into(),
0, len);
BooleanArray::new(boolean_buf, self.nulls)
}
Err(buf) => {
let mut builder = BooleanBufferBuilder::new(len);
builder.append_buffer(&BooleanBuffer::new(buf, 0, end));
builder.append_n(len - end, false);
BooleanArray::new(builder.finish(), self.nulls)
}
}
```
essentially rewrite at the byte level, except for if the `end` was inside a
byte so we need to do some bit ops there
i did a single benchmark run and it seems promising, though i havent
carefully checked for edge cases yet 🤔
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
