adriangb commented on PR #18449:
URL: https://github.com/apache/datafusion/pull/18449#issuecomment-3509238908
@alamb could you run the benchmark again? I'm getting promising results
locally:
```
in_list_utf8(5) (1024, 0) IN (1, 0)
time: [2.1329 µs 2.1369 µs 2.1412 µs]
change: [−24.304% −24.100% −23.908%] (p = 0.00 <
0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) high mild
2 (2.00%) high severe
in_list_utf8(10) (1024, 0) IN (1, 0)
time: [2.1459 µs 2.1495 µs 2.1533 µs]
change: [−23.928% −23.660% −23.351%] (p = 0.00 <
0.05)
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
in_list_utf8(20) (1024, 0) IN (1, 0)
time: [2.1438 µs 2.1482 µs 2.1529 µs]
change: [−35.000% −32.830% −30.673%] (p = 0.00 <
0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) high mild
1 (1.00%) high severe
in_list_f32 (1024, 0) IN (1, 0)
time: [2.1289 µs 2.1326 µs 2.1368 µs]
change: [−38.413% −36.728% −35.228%] (p = 0.00 <
0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severe
in_list_i32 (1024, 0) IN (1, 0)
time: [2.1273 µs 2.1322 µs 2.1381 µs]
change: [−16.900% −16.602% −16.305%] (p = 0.00 <
0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) low mild
1 (1.00%) high mild
1 (1.00%) high severe
in_list_utf8(5) (1024, 0.2) IN (1, 0)
time: [3.0365 µs 3.0428 µs 3.0490 µs]
change: [−36.487% −34.532% −32.458%] (p = 0.00 <
0.05)
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
in_list_utf8(10) (1024, 0.2) IN (1, 0)
time: [3.0444 µs 3.0522 µs 3.0595 µs]
change: [−36.231% −34.959% −33.607%] (p = 0.00 <
0.05)
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
in_list_utf8(20) (1024, 0.2) IN (1, 0)
time: [3.1244 µs 3.1314 µs 3.1385 µs]
change: [−11.989% −11.717% −11.422%] (p = 0.00 <
0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
1 (1.00%) low mild
3 (3.00%) high mild
1 (1.00%) high severe
in_list_f32 (1024, 0.2) IN (1, 0)
time: [2.8616 µs 2.8686 µs 2.8761 µs]
change: [−4.6147% −4.1434% −3.6780%] (p = 0.00 <
0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
2 (2.00%) high mild
3 (3.00%) high severe
in_list_i32 (1024, 0.2) IN (1, 0)
time: [2.8689 µs 2.8800 µs 2.8948 µs]
change: [−6.4977% −6.0731% −5.5574%] (p = 0.00 <
0.05)
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high severe
in_list_utf8(5) (1024, 0) IN (3, 0)
time: [2.1316 µs 2.1376 µs 2.1439 µs]
change: [−24.605% −24.365% −24.166%] (p = 0.00 <
0.05)
Performance has improved.
in_list_utf8(10) (1024, 0) IN (3, 0)
time: [2.1284 µs 2.1326 µs 2.1368 µs]
change: [−25.190% −24.964% −24.739%] (p = 0.00 <
0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severe
in_list_utf8(20) (1024, 0) IN (3, 0)
time: [2.1289 µs 2.1335 µs 2.1384 µs]
change: [−24.859% −24.619% −24.366%] (p = 0.00 <
0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe
in_list_f32 (1024, 0) IN (3, 0)
time: [2.1245 µs 2.1288 µs 2.1334 µs]
change: [−15.609% −15.371% −15.158%] (p = 0.00 <
0.05)
Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
7 (7.00%) high mild
2 (2.00%) high severe
in_list_i32 (1024, 0) IN (3, 0)
time: [2.1255 µs 2.1299 µs 2.1343 µs]
change: [−15.936% −15.621% −15.328%] (p = 0.00 <
0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe
in_list_utf8(5) (1024, 0.2) IN (3, 0)
time: [3.0553 µs 3.0637 µs 3.0724 µs]
change: [−15.090% −14.743% −14.427%] (p = 0.00 <
0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
1 (1.00%) low mild
3 (3.00%) high mild
1 (1.00%) high severe
in_list_utf8(10) (1024, 0.2) IN (3, 0)
time: [3.1270 µs 3.1356 µs 3.1447 µs]
change: [−11.801% −11.394% −11.027%] (p = 0.00 <
0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe
in_list_utf8(20) (1024, 0.2) IN (3, 0)
time: [3.1550 µs 3.1655 µs 3.1788 µs]
change: [−13.039% −12.625% −12.188%] (p = 0.00 <
0.05)
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high severe
in_list_f32 (1024, 0.2) IN (3, 0)
time: [2.9158 µs 2.9245 µs 2.9333 µs]
change: [−3.8833% −3.2912% −2.7119%] (p = 0.00 <
0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
in_list_i32 (1024, 0.2) IN (3, 0)
time: [2.8837 µs 2.8928 µs 2.9035 µs]
change: [−5.0262% −4.6043% −4.1990%] (p = 0.00 <
0.05)
Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
1 (1.00%) low severe
7 (7.00%) high mild
1 (1.00%) high severe
in_list_utf8(5) (1024, 0) IN (10, 0)
time: [2.1303 µs 2.1489 µs 2.1734 µs]
change: [−24.301% −21.837% −17.687%] (p = 0.00 <
0.05)
Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
1 (1.00%) high mild
9 (9.00%) high severe
in_list_utf8(10) (1024, 0) IN (10, 0)
time: [2.1435 µs 2.1549 µs 2.1756 µs]
change: [−24.680% −24.110% −23.317%] (p = 0.00 <
0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
1 (1.00%) low mild
1 (1.00%) high mild
3 (3.00%) high severe
in_list_utf8(20) (1024, 0) IN (10, 0)
time: [2.1390 µs 2.1454 µs 2.1522 µs]
change: [−24.712% −24.486% −24.246%] (p = 0.00 <
0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
7 (7.00%) high mild
1 (1.00%) high severe
in_list_f32 (1024, 0) IN (10, 0)
time: [2.0991 µs 2.1036 µs 2.1088 µs]
change: [−15.814% −15.569% −15.301%] (p = 0.00 <
0.05)
Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
7 (7.00%) high mild
2 (2.00%) high severe
in_list_i32 (1024, 0) IN (10, 0)
time: [2.1197 µs 2.1248 µs 2.1301 µs]
change: [−15.648% −15.282% −14.783%] (p = 0.00 <
0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe
in_list_utf8(5) (1024, 0.2) IN (10, 0)
time: [3.0688 µs 3.0835 µs 3.1023 µs]
change: [−15.752% −14.224% −13.106%] (p = 0.00 <
0.05)
Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
3 (3.00%) high mild
3 (3.00%) high severe
in_list_utf8(10) (1024, 0.2) IN (10, 0)
time: [3.1224 µs 3.1369 µs 3.1606 µs]
change: [−10.942% −10.579% −10.179%] (p = 0.00 <
0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severe
in_list_utf8(20) (1024, 0.2) IN (10, 0)
time: [3.1560 µs 3.1706 µs 3.1961 µs]
change: [−13.605% −13.319% −12.929%] (p = 0.00 <
0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) low mild
1 (1.00%) high mild
2 (2.00%) high severe
in_list_f32 (1024, 0.2) IN (10, 0)
time: [2.9480 µs 2.9641 µs 2.9883 µs]
change: [−1.7084% −1.0839% −0.4270%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
1 (1.00%) low mild
5 (5.00%) high mild
2 (2.00%) high severe
in_list_i32 (1024, 0.2) IN (10, 0)
time: [2.9054 µs 2.9144 µs 2.9238 µs]
change: [−4.3963% −4.0561% −3.7253%] (p = 0.00 <
0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
in_list_utf8(5) (1024, 0) IN (100, 0)
time: [2.1311 µs 2.1366 µs 2.1430 µs]
change: [−30.391% −27.461% −25.061%] (p = 0.00 <
0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
3 (3.00%) high mild
5 (5.00%) high severe
in_list_utf8(10) (1024, 0) IN (100, 0)
time: [2.1301 µs 2.1408 µs 2.1555 µs]
change: [−24.344% −23.667% −22.620%] (p = 0.00 <
0.05)
Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
2 (2.00%) high mild
4 (4.00%) high severe
in_list_utf8(20) (1024, 0) IN (100, 0)
time: [2.1389 µs 2.1443 µs 2.1494 µs]
change: [−24.752% −24.532% −24.303%] (p = 0.00 <
0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severe
in_list_f32 (1024, 0) IN (100, 0)
time: [2.1481 µs 2.1567 µs 2.1676 µs]
change: [−14.858% −14.459% −14.017%] (p = 0.00 <
0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high severe
in_list_i32 (1024, 0) IN (100, 0)
time: [2.1556 µs 2.1655 µs 2.1786 µs]
change: [−14.130% −13.702% −13.179%] (p = 0.00 <
0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
1 (1.00%) high mild
2 (2.00%) high severe
in_list_utf8(5) (1024, 0.2) IN (100, 0)
time: [3.3774 µs 3.4003 µs 3.4289 µs]
change: [−2.1519% −1.4661% −0.6108%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
4 (4.00%) high severe
in_list_utf8(10) (1024, 0.2) IN (100, 0)
time: [3.1737 µs 3.1907 µs 3.2192 µs]
change: [−10.181% −9.7063% −9.1830%] (p = 0.00 <
0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
4 (4.00%) high mild
4 (4.00%) high severe
in_list_utf8(20) (1024, 0.2) IN (100, 0)
time: [3.3150 µs 3.3270 µs 3.3424 µs]
change: [−7.7706% −6.3142% −5.1134%] (p = 0.00 <
0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
1 (1.00%) low mild
1 (1.00%) high mild
3 (3.00%) high severe
in_list_f32 (1024, 0.2) IN (100, 0)
time: [3.6284 µs 3.7404 µs 3.8540 µs]
change: [+12.781% +15.878% +18.866%] (p = 0.00 <
0.05)
Performance has regressed.
in_list_i32 (1024, 0.2) IN (100, 0)
time: [3.7092 µs 3.7750 µs 3.8491 µs]
change: [+25.967% +28.595% +31.510%] (p = 0.00 <
0.05)
Performance has regressed.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
