alamb commented on code in PR #10229:
URL: https://github.com/apache/arrow-rs/pull/10229#discussion_r3498648114
##########
arrow-row/src/fixed.rs:
##########
@@ -402,61 +397,19 @@ pub fn decode_bool(rows: &mut [&[u8]], options:
SortOptions) -> BooleanArray {
values.push(values_packed);
}
- let builder = ArrayDataBuilder::new(DataType::Boolean)
- .len(rows.len())
- .null_count(null_count)
- .add_buffer(values.into())
- .null_bit_buffer(Some(nulls.into()));
+ let nulls = NullBuffer::new(BooleanBuffer::new(nulls.into(), 0, len));
+ let nulls = (nulls.null_count() > 0).then_some(nulls);
- // SAFETY:
- // Buffers are the correct length
- unsafe { BooleanArray::from(builder.build_unchecked()) }
+ BooleanArray::new(BooleanBuffer::new(values.into(), 0, len), nulls)
}
/// Decodes a single byte from each row, interpreting `0x01` as a valid value
-/// and all other values as a null
-///
-/// Returns the null count and null buffer
-pub fn decode_nulls(rows: &[&[u8]]) -> (usize, Buffer) {
Review Comment:
Yes -- when we last checked LLVM was using the native hardware instruction
to count set bits which was crazy fast
As I recall there were a few times it was faster to avoid a second pass
through the data, but only when the bits were already being processed a word
(`u64`) at a time, not individually
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
