AntoinePrv commented on code in PR #46789:
URL: https://github.com/apache/arrow/pull/46789#discussion_r2161895195
##########
cpp/src/arrow/util/byte_stream_split_internal.h:
##########
@@ -123,95 +185,79 @@ void ByteStreamSplitEncodeSimd128(const uint8_t*
raw_values, int width,
output_buffer_raw[j * num_values + i] = byte_in_value;
}
}
- // The current shuffling algorithm diverges for float and double types but
the compiler
- // should be able to remove the branch since only one path is taken for each
template
- // instantiation.
- // Example run for 32-bit variables:
- // Step 0: copy from unaligned input bytes:
- // 0: ABCD ABCD ABCD ABCD 1: ABCD ABCD ABCD ABCD ...
- // Step 1: simd_batch<int8_t, 8>::zip_lo and simd_batch<int8_t, 8>::zip_hi:
- // 0: AABB CCDD AABB CCDD 1: AABB CCDD AABB CCDD ...
- // Step 2: apply simd_batch<int8_t, 8>::zip_lo and simd_batch<int8_t,
8>::zip_hi again:
- // 0: AAAA BBBB CCCC DDDD 1: AAAA BBBB CCCC DDDD ...
- // Step 3: simd_batch<int8_t, 8>::zip_lo and simd_batch<int8_t, 8>::zip_hi:
- // 0: AAAA AAAA BBBB BBBB 1: CCCC CCCC DDDD DDDD ...
- // Step 4: simd_batch<int64_t, 2>::zip_lo and simd_batch<int64_t, 2>::zip_hi:
- // 0: AAAA AAAA AAAA AAAA 1: BBBB BBBB BBBB BBBB ...
+
+ // Number of input values we can fit in a simd register
+ constexpr int kNumValuesInBatch = simd_batch::size / kNumStreams;
+ static_assert(kNumValuesInBatch > 0);
+ // Number of bytes we'll bring together in the first byte-level part of the
algorithm.
+ // Since we zip with the next batch, the number of values in a batch
determines how many
+ // bytes end up together before we can use a larger type
+ constexpr int kNumBytes = 2 * kNumValuesInBatch;
+ // Number of steps in the first part of the algorithm with byte-level zipping
+ constexpr int kNumStepsByte = ReversePow2(kNumValuesInBatch) + 1;
+ // Number of steps in the first part of the algorithm with large data type
zipping
+ constexpr int kNumStepsLarge =
+ ReversePow2(static_cast<int>(simd_batch::size) / kNumBytes);
+ // Total number of steps
+ constexpr int kNumSteps = kNumStepsByte + kNumStepsLarge;
Review Comment:
Correct, I did not see it that way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
