AdamGS commented on PR #9625:
URL: https://github.com/apache/arrow-rs/pull/9625#issuecomment-4154021338
Running the `take_kernels` benchmark locally (MacBook Pro, M3 Max), I get
the following results, seems like the non-null cases have some variability in
the results, across multiple runs they are +/-3%
```
Benchmarking take str 512: Collecting 100 samples in estimated 5.0094 s
(1.7M iteratiotake str 512 time: [2.9368 µs 2.9759 µs 3.0239 µs]
change: [+2.2019% +3.8482% +5.6373%] (p = 0.00 <
0.05)
Performance has regressed.
Found 21 outliers among 100 measurements (21.00%)
19 (19.00%) high mild
2 (2.00%) high severe
Benchmarking take str 1024: Collecting 100 samples in estimated 5.0295 s
(752k iteratitake str 1024 time: [6.8638 µs 6.9733 µs 7.0946 µs]
change: [+1.5596% +2.8349% +4.2869%] (p = 0.00 <
0.05)
Performance has regressed.
Benchmarking take str null indices 512: Collecting 100 samples in estimated
5.0064 s (take str null indices 512
time: [1.7367 µs 1.7383 µs 1.7401 µs]
change: [−12.578% −12.355% −12.130%] (p = 0.00 <
0.05)
Performance has improved.
Found 14 outliers among 100 measurements (14.00%)
1 (1.00%) low severe
1 (1.00%) low mild
10 (10.00%) high mild
2 (2.00%) high severe
Benchmarking take str null indices 1024: Collecting 100 samples in estimated
5.0038 s take str null indices 1024
time: [4.1551 µs 4.1610 µs 4.1670 µs]
change: [−5.7784% −5.6160% −5.4566%] (p = 0.00 <
0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
3 (3.00%) low severe
1 (1.00%) low mild
3 (3.00%) high mild
1 (1.00%) high severe
Benchmarking take str null values 1024: Collecting 100 samples in estimated
5.0192 s (take str null values 1024
time: [4.4388 µs 4.4466 µs 4.4537 µs]
change: [−12.466% −12.197% −11.916%] (p = 0.00 <
0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
2 (2.00%) low severe
1 (1.00%) low mild
1 (1.00%) high mild
1 (1.00%) high severe
Benchmarking take str null values null indices 1024: Collecting 100 samples
in estimattake str null values null indices 1024
time: [2.7090 µs 2.7227 µs 2.7356 µs]
change: [−26.967% −26.737% −26.506%] (p = 0.00 <
0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
4 (4.00%) high mild
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
