jhorstmann commented on a change in pull request #8262:
URL: https://github.com/apache/arrow/pull/8262#discussion_r501769149
##########
File path: rust/arrow/src/buffer.rs
##########
@@ -369,120 +394,171 @@ where
result.freeze()
}
+/// Apply a bitwise operation `op` to two inputs and return the result as a
Buffer.
+/// The inputs are treated as bitmaps, meaning that offsets and length are
specified in number of bits.
fn bitwise_bin_op_helper<F>(
left: &Buffer,
- left_offset: usize,
+ left_offset_in_bits: usize,
right: &Buffer,
- right_offset: usize,
- len: usize,
+ right_offset_in_bits: usize,
+ len_in_bits: usize,
op: F,
) -> Buffer
where
- F: Fn(u8, u8) -> u8,
+ F: Fn(u64, u64) -> u64,
{
- let mut result = MutableBuffer::new(len).with_bitset(len, false);
+ // reserve capacity and set length so we can get a typed view of u64 chunks
+ let mut result =
+ MutableBuffer::new(ceil(len_in_bits, 8)).with_bitset(len_in_bits / 64
* 8, false);
- result
- .data_mut()
- .iter_mut()
- .zip(
- left.data()[left_offset..]
- .iter()
- .zip(right.data()[right_offset..].iter()),
- )
+ let left_chunks = left.bit_chunks(left_offset_in_bits, len_in_bits);
+ let right_chunks = right.bit_chunks(right_offset_in_bits, len_in_bits);
+ let result_chunks = result.typed_data_mut::<u64>().iter_mut();
+
+ result_chunks
+ .zip(left_chunks.iter().zip(right_chunks.iter()))
Review comment:
Yes, that sounds correct. The result buffer is newly created and
properly aligned for the largest primitive types.
The `with_bitset` call sets the len of the buffer to multiple of 8bytes /
64bits, as otherwise the `typed_data_mut` function would complain. The capacity
of the buffer is however large enough to also contain the remainder bytes.
There are some comparison opcodes left in the assembly, I couldn't find a
way to convince the compiler that all iterators have the same length. Still it
seems to be not much slower than the simd version.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
