etseidl commented on PR #6204:
URL: https://github.com/apache/arrow-rs/pull/6204#issuecomment-2272324462
Example usage and output:
```
% cargo bench -p parquet --bench encoding --all-features
Finished `bench` profile [optimized] target(s) in 0.71s
Running benches/encoding.rs
(target/release/deps/encoding-fa07f9604f013e9b)
encoding: dtype=f32, encoding=BYTE_STREAM_SPLIT
time: [44.223 µs 44.519 µs 44.860 µs]
change: [-1.0311% +0.0307% +1.0725%] (p = 0.95 >
0.05)
No change in performance detected.
dtype=f32, encoding=BYTE_STREAM_SPLIT encoded as 65536 bytes
decoding: dtype=f32, encoding=BYTE_STREAM_SPLIT
time: [38.790 µs 38.997 µs 39.257 µs]
change: [-5.5214% -1.7711% +1.2431%] (p = 0.36 >
0.05)
No change in performance detected.
Found 15 outliers among 100 measurements (15.00%)
4 (4.00%) high mild
11 (11.00%) high severe
encoding: dtype=f64, encoding=BYTE_STREAM_SPLIT
time: [108.12 µs 108.47 µs 108.85 µs]
change: [-54.122% -46.682% -39.381%] (p = 0.00 <
0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
4 (4.00%) high mild
4 (4.00%) high severe
dtype=f64, encoding=BYTE_STREAM_SPLIT encoded as 131072 bytes
decoding: dtype=f64, encoding=BYTE_STREAM_SPLIT
time: [81.265 µs 81.526 µs 81.789 µs]
change: [-10.273% -9.2926% -8.3289%] (p = 0.00 <
0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) high mild
3 (3.00%) high severe
encoding: dtype=parquet::data_type::FixedLenByteArray,
encoding=BYTE_STREAM_SPLIT
time: [46.957 µs 47.171 µs 47.437 µs]
change: [-26.632% -23.632% -20.584%] (p = 0.00 <
0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) high mild
1 (1.00%) high severe
dtype=parquet::data_type::FixedLenByteArray, encoding=BYTE_STREAM_SPLIT
encoded as 32768 bytes
decoding: dtype=parquet::data_type::FixedLenByteArray,
encoding=BYTE_STREAM_SPLIT
time: [384.39 µs 385.17 µs 386.00 µs]
change: [-11.955% -10.898% -9.8426%] (p = 0.00 <
0.05)
Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
5 (5.00%) high mild
5 (5.00%) high severe
encoding: dtype=parquet::data_type::FixedLenByteArray,
encoding=BYTE_STREAM_SPLIT #2
time: [270.75 µs 274.65 µs 278.81 µs]
change: [-2.4969% -0.9976% +0.6427%] (p = 0.24 >
0.05)
No change in performance detected.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
dtype=parquet::data_type::FixedLenByteArray, encoding=BYTE_STREAM_SPLIT
encoded as 262144 bytes
decoding: dtype=parquet::data_type::FixedLenByteArray,
encoding=BYTE_STREAM_SPLIT #2
time: [542.87 µs 543.70 µs 544.61 µs]
change: [-16.128% -13.963% -11.712%] (p = 0.00 <
0.05)
Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
6 (6.00%) high severe
```
```
% cargo bench -p parquet --bench arrow_reader --all-features -- BYTE_STREAM
Finished `bench` profile [optimized] target(s) in 0.31s
Running benches/arrow_reader.rs
(target/release/deps/arrow_reader-f970ea21ed6b3164)
arrow_array_reader/BYTE_STREAM_SPLIT/Decimal128Array/byte_stream_split
encoded, mandatory, no NULLs
time: [3.4332 ms 3.4747 ms 3.5191 ms]
change: [+4.5989% +5.7616% +7.1541%] (p = 0.00 <
0.05)
Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild
arrow_array_reader/BYTE_STREAM_SPLIT/Decimal128Array/byte_stream_split
encoded, optional, no NULLs
time: [3.3593 ms 3.3818 ms 3.4055 ms]
change: [-5.2748% -4.1025% -2.8714%] (p = 0.00 <
0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
5 (5.00%) high mild
arrow_array_reader/BYTE_STREAM_SPLIT/Decimal128Array/byte_stream_split
encoded, optional, half NULLs
time: [3.0576 ms 3.0768 ms 3.0976 ms]
change: [-5.6299% -4.1956% -2.8196%] (p = 0.00 <
0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
arrow_array_reader/BYTE_STREAM_SPLIT/Float16Array/byte_stream_split encoded,
mandatory, no NULLs
time: [823.94 µs 827.82 µs 832.00 µs]
change: [-5.9189% -4.2386% -2.7745%] (p = 0.00 <
0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe
arrow_array_reader/BYTE_STREAM_SPLIT/Float16Array/byte_stream_split encoded,
optional, no NULLs
time: [845.52 µs 857.75 µs 873.06 µs]
change: [-2.3817% -1.0937% +0.1853%] (p = 0.10 >
0.05)
No change in performance detected.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe
arrow_array_reader/BYTE_STREAM_SPLIT/Float16Array/byte_stream_split encoded,
optional, half NULLs
time: [1.5441 ms 1.5657 ms 1.5884 ms]
change: [+1.2815% +2.5329% +3.8183%] (p = 0.00 <
0.05)
Performance has regressed.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
