alamb commented on PR #9120:
URL: https://github.com/apache/arrow-rs/pull/9120#issuecomment-3734529589
> `arrow_reader_clickbench/async/Q20 1.21 149.2±1.53ms ? ?/sec 1.00
123.3±0.69ms ? ?/sec`
>
> This "regression" comes up twice in a row?
I was able to reproduce a smaller regression (about 1%) -- I'll see what I
can find
```shell
cargo bench --features="arrow async" --bench arrow_reader_clickbench -- Q20
```
Here it is with `git merge-base HEAD apache/main` (aka where the branch
diverged from main):
* sync: 89.911 ms / async: 59.136ms
With this PR:
* sync 90.376 ms / async: 60.336 ms
<details><summary>Details</summary>
<p>
```
arrow_reader_clickbench/sync/Q20
time: [89.676 ms 89.911 ms 90.206 ms]
change: [−0.9129% −0.4274% +0.0319%] (p = 0.08 >
0.05)
No change in performance detected.
Found 9 outliers among 100 measurements (9.00%)
3 (3.00%) high mild
6 (6.00%) high severe
Benchmarking arrow_reader_clickbench/async/Q20: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 5.9s, or reduce sample count to 80.
arrow_reader_clickbench/async/Q20
time: [59.034 ms 59.136 ms 59.246 ms]
change: [−0.3050% −0.0199% +0.2437%] (p = 0.89 >
0.05)
No change in performance detected.
Found 12 outliers among 100 measurements (12.00%)
7 (7.00%) high mild
5 (5.00%) high severe
(venv) andrewlamb@Andrews-MacBook-Pro-3:~/Software/arrow-rs$
```
Here it is on this branch
```
arrow_reader_clickbench/sync/Q20
time: [90.004 ms 90.376 ms 90.785 ms]
change: [+0.0164% +0.5169% +1.0574%] (p = 0.06 >
0.05)
No change in performance detected.
Found 14 outliers among 100 measurements (14.00%)
6 (6.00%) high mild
8 (8.00%) high severe
Benchmarking arrow_reader_clickbench/async/Q20: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 6.0s, or reduce sample count to 80.
arrow_reader_clickbench/async/Q20
time: [60.147 ms 60.336 ms 60.542 ms]
change: [+1.6210% +2.0286% +2.4391%] (p = 0.00 <
0.05)
Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
```
</p>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
