etseidl commented on PR #9447:
URL: https://github.com/apache/arrow-rs/pull/9447#issuecomment-4093283650
Ahh, I get it. The benchbot can't handle the remote branch also being named
"main".
Not much difference either way on my laptop
<details>
```
group 9447
main
----- ----
----
bool/bloom_filter 1.00 157.7±1.83µs 6.7
MB/sec 1.00 157.4±2.00µs 6.7 MB/sec
bool/default 1.01 67.5±2.00µs 15.7
MB/sec 1.00 66.6±1.55µs 15.9 MB/sec
bool/parquet_2 1.01 83.1±1.88µs 12.8
MB/sec 1.00 82.1±1.65µs 12.9 MB/sec
bool/zstd 1.00 78.5±1.82µs 13.5
MB/sec 1.01 79.6±2.78µs 13.3 MB/sec
bool/zstd_parquet_2 1.00 92.3±1.55µs 11.5
MB/sec 1.01 92.9±1.71µs 11.4 MB/sec
bool_non_null/bloom_filter 1.00 121.9±1.31µs 4.7
MB/sec 1.01 122.6±1.68µs 4.7 MB/sec
bool_non_null/default 1.00 29.4±0.48µs 19.4
MB/sec 1.00 29.4±0.64µs 19.5 MB/sec
bool_non_null/parquet_2 1.00 46.0±0.73µs 12.5
MB/sec 1.01 46.6±1.04µs 12.3 MB/sec
bool_non_null/zstd 1.00 39.0±0.59µs 14.7
MB/sec 1.01 39.5±0.76µs 14.5 MB/sec
bool_non_null/zstd_parquet_2 1.00 56.5±1.05µs 10.1
MB/sec 1.01 57.0±1.07µs 10.0 MB/sec
float_with_nans/bloom_filter 1.01 1100.0±21.00µs 50.0
MB/sec 1.00 1090.2±15.25µs 50.4 MB/sec
float_with_nans/default 1.00 712.1±20.32µs 77.2
MB/sec 1.01 718.0±28.49µs 76.5 MB/sec
float_with_nans/parquet_2 1.01 1001.0±30.26µs 54.9
MB/sec 1.00 995.0±24.92µs 55.2 MB/sec
float_with_nans/zstd 1.00 907.4±20.16µs 60.6
MB/sec 1.00 902.9±15.18µs 60.9 MB/sec
float_with_nans/zstd_parquet_2 1.01 1190.0±32.39µs 46.2
MB/sec 1.00 1181.1±25.41µs 46.5 MB/sec
list_primitive/bloom_filter 1.02 2.9±0.04ms 736.6
MB/sec 1.00 2.8±0.06ms 750.1 MB/sec
list_primitive/default 1.00 2.0±0.05ms 1041.7
MB/sec 1.00 2.0±0.04ms 1044.9 MB/sec
list_primitive/parquet_2 1.00 2.1±0.04ms 1036.1
MB/sec 1.00 2.1±0.04ms 1039.4 MB/sec
list_primitive/zstd 1.00 3.5±0.06ms 614.6
MB/sec 1.00 3.5±0.08ms 614.2 MB/sec
list_primitive/zstd_parquet_2 1.00 3.6±0.07ms 593.3
MB/sec 1.01 3.6±0.07ms 585.1 MB/sec
list_primitive_non_null/bloom_filter 1.00 3.4±0.17ms 627.5
MB/sec 1.10 3.7±0.10ms 570.8 MB/sec
list_primitive_non_null/default 1.00 2.2±0.06ms 953.4
MB/sec 1.00 2.2±0.05ms 950.7 MB/sec
list_primitive_non_null/parquet_2 1.15 3.0±0.07ms 716.4
MB/sec 1.00 2.6±0.06ms 826.1 MB/sec
list_primitive_non_null/zstd 1.00 6.4±0.12ms 332.0
MB/sec 1.01 6.5±0.29ms 328.3 MB/sec
list_primitive_non_null/zstd_parquet_2 1.01 6.4±0.13ms 330.2
MB/sec 1.00 6.4±0.15ms 331.9 MB/sec
primitive/bloom_filter 1.01 1688.0±39.53µs 104.2
MB/sec 1.00 1668.9±38.29µs 105.4 MB/sec
primitive/default 1.00 863.1±15.45µs 203.8
MB/sec 1.00 861.5±18.26µs 204.2 MB/sec
primitive/parquet_2 1.00 863.8±14.79µs 203.7
MB/sec 1.00 867.1±29.50µs 202.9 MB/sec
primitive/zstd 1.00 1227.9±24.67µs 143.3
MB/sec 1.00 1230.7±30.77µs 143.0 MB/sec
primitive/zstd_parquet_2 1.00 1273.0±31.28µs 138.2
MB/sec 1.00 1271.6±38.95µs 138.4 MB/sec
primitive_non_null/bloom_filter 1.00 1561.1±35.74µs 110.5
MB/sec 1.00 1555.0±39.18µs 110.9 MB/sec
primitive_non_null/default 1.00 670.8±20.08µs 257.2
MB/sec 1.00 671.3±13.90µs 257.0 MB/sec
primitive_non_null/parquet_2 1.00 674.1±22.93µs 255.9
MB/sec 1.00 676.5±13.99µs 255.0 MB/sec
primitive_non_null/zstd 1.01 1070.7±27.28µs 161.1
MB/sec 1.00 1064.8±31.81µs 162.0 MB/sec
primitive_non_null/zstd_parquet_2 1.00 1151.1±49.02µs 149.9
MB/sec 1.01 1159.6±46.01µs 148.8 MB/sec
string/bloom_filter 1.01 1641.3±35.39µs 1247.8
MB/sec 1.00 1628.6±32.02µs 1257.6 MB/sec
string/default 1.00 1025.0±24.08µs 1998.1
MB/sec 1.00 1025.1±30.04µs 1998.0 MB/sec
string/parquet_2 1.00 1028.6±28.34µs 1991.1
MB/sec 1.01 1039.8±35.29µs 1969.7 MB/sec
string/zstd 1.00 2.8±0.06ms 719.8
MB/sec 1.00 2.8±0.06ms 721.0 MB/sec
string/zstd_parquet_2 1.00 3.2±0.16ms 636.7
MB/sec 1.01 3.2±0.07ms 632.3 MB/sec
string_and_binary_view/bloom_filter 1.00 739.4±21.56µs 170.7
MB/sec 1.00 737.5±33.35µs 171.1 MB/sec
string_and_binary_view/default 1.01 473.7±12.23µs 266.4
MB/sec 1.00 468.0±8.82µs 269.6 MB/sec
string_and_binary_view/parquet_2 1.01 477.5±7.91µs 264.3
MB/sec 1.00 472.2±8.99µs 267.2 MB/sec
string_and_binary_view/zstd 1.01 765.5±27.71µs 164.8
MB/sec 1.00 757.7±12.94µs 166.5 MB/sec
string_and_binary_view/zstd_parquet_2 1.01 746.7±13.67µs 169.0
MB/sec 1.00 742.1±14.61µs 170.0 MB/sec
string_dictionary/bloom_filter 1.02 854.5±26.17µs 1207.8
MB/sec 1.00 835.9±14.69µs 1234.6 MB/sec
string_dictionary/default 1.01 526.6±15.16µs 1959.9
MB/sec 1.00 522.1±19.22µs 1976.6 MB/sec
string_dictionary/parquet_2 1.03 539.7±23.54µs 1912.1
MB/sec 1.00 521.7±16.10µs 1978.3 MB/sec
string_dictionary/zstd 1.04 1629.5±96.16µs 633.3
MB/sec 1.00 1564.0±35.74µs 659.9 MB/sec
string_dictionary/zstd_parquet_2 1.04 1451.9±78.11µs 710.8
MB/sec 1.00 1399.6±31.57µs 737.3 MB/sec
string_non_null/bloom_filter 1.00 2.3±0.12ms 894.3
MB/sec 1.03 2.3±0.07ms 871.6 MB/sec
string_non_null/default 1.04 1525.9±103.77µs 1341.6
MB/sec 1.00 1471.2±71.91µs 1391.5 MB/sec
string_non_null/parquet_2 1.00 1440.6±52.50µs 1421.1
MB/sec 1.01 1459.1±57.98µs 1403.0 MB/sec
string_non_null/zstd 1.00 4.0±0.12ms 511.3
MB/sec 1.01 4.1±0.12ms 505.4 MB/sec
string_non_null/zstd_parquet_2 1.00 4.2±0.14ms 491.8
MB/sec 1.00 4.2±0.11ms 492.3 MB/sec
```
</details>
On my WS the times are all over the place :( The huge regressions can show
up on either branch (could be too much else going on during benchmarking)
<details>
```
group 9447
main
----- ----
----
bool/bloom_filter 1.01 61.7±0.24µs 17.2
MB/sec 1.00 61.3±0.59µs 17.3 MB/sec
bool/default 1.01 27.0±0.10µs 39.2
MB/sec 1.00 26.9±0.28µs 39.4 MB/sec
bool/parquet_2 1.02 31.2±0.31µs 34.0
MB/sec 1.00 30.7±0.21µs 34.5 MB/sec
bool/zstd 1.01 32.5±0.23µs 32.6
MB/sec 1.00 32.4±0.36µs 32.8 MB/sec
bool/zstd_parquet_2 1.01 35.9±0.21µs 29.5
MB/sec 1.00 35.6±0.67µs 29.8 MB/sec
bool_non_null/bloom_filter 1.01 52.0±0.37µs 11.0
MB/sec 1.00 51.5±0.45µs 11.1 MB/sec
bool_non_null/default 1.00 12.5±0.09µs 45.8
MB/sec 1.00 12.5±0.07µs 45.8 MB/sec
bool_non_null/parquet_2 1.01 17.8±0.06µs 32.1
MB/sec 1.00 17.6±0.24µs 32.5 MB/sec
bool_non_null/zstd 1.01 17.4±0.27µs 32.9
MB/sec 1.00 17.1±0.08µs 33.4 MB/sec
bool_non_null/zstd_parquet_2 1.02 22.6±0.09µs 25.3
MB/sec 1.00 22.2±0.24µs 25.7 MB/sec
float_with_nans/bloom_filter 1.00 412.4±6.75µs 133.3
MB/sec 1.00 414.5±1.54µs 132.6 MB/sec
float_with_nans/default 1.00 243.8±2.62µs 225.5
MB/sec 1.02 247.9±1.34µs 221.7 MB/sec
float_with_nans/parquet_2 1.00 383.7±1.83µs 143.2
MB/sec 1.02 391.6±5.04µs 140.4 MB/sec
float_with_nans/zstd 1.00 354.7±3.99µs 155.0
MB/sec 1.01 357.4±1.57µs 153.8 MB/sec
float_with_nans/zstd_parquet_2 1.00 490.2±5.70µs 112.1
MB/sec 1.02 498.8±2.48µs 110.2 MB/sec
list_primitive/bloom_filter 1.00 1256.3±15.07µs 1697.0
MB/sec 1.56 1955.6±9.67µs 1090.1 MB/sec
list_primitive/default 1.00 948.9±5.49µs 2.2
GB/sec 1.03 975.8±20.29µs 2.1 GB/sec
list_primitive/parquet_2 1.00 970.7±3.47µs 2.1
GB/sec 1.32 1282.9±17.01µs 1661.8 MB/sec
list_primitive/zstd 1.00 1611.5±16.26µs 1322.9
MB/sec 1.00 1611.3±9.27µs 1323.1 MB/sec
list_primitive/zstd_parquet_2 1.00 1623.5±8.28µs 1313.2
MB/sec 1.22 1977.4±17.31µs 1078.1 MB/sec
list_primitive_non_null/bloom_filter 1.00 1429.8±25.68µs 1487.9
MB/sec 1.54 2.2±0.01ms 965.6 MB/sec
list_primitive_non_null/default 1.00 972.8±6.84µs 2.1
GB/sec 1.02 992.6±12.82µs 2.1 GB/sec
list_primitive_non_null/parquet_2 1.00 1049.3±7.22µs 2027.3
MB/sec 1.37 1438.6±17.86µs 1478.8 MB/sec
list_primitive_non_null/zstd 1.00 2.1±0.03ms 1009.0
MB/sec 1.21 2.6±0.01ms 831.4 MB/sec
list_primitive_non_null/zstd_parquet_2 1.00 2.1±0.03ms 998.8
MB/sec 1.32 2.8±0.05ms 759.5 MB/sec
primitive/bloom_filter 1.02 1739.9±56.99µs 101.1
MB/sec 1.00 1701.9±32.84µs 103.4 MB/sec
primitive/default 1.02 351.6±1.71µs 500.4
MB/sec 1.00 344.4±4.48µs 510.9 MB/sec
primitive/parquet_2 1.05 402.0±3.50µs 437.7
MB/sec 1.00 382.7±12.07µs 459.7 MB/sec
primitive/zstd 1.02 512.4±6.91µs 343.4
MB/sec 1.00 502.7±4.61µs 350.0 MB/sec
primitive/zstd_parquet_2 1.00 480.7±2.43µs 366.0
MB/sec 1.24 596.9±7.32µs 294.8 MB/sec
primitive_non_null/bloom_filter 1.00 690.2±11.84µs 250.0
MB/sec 2.42 1671.9±9.10µs 103.2 MB/sec
primitive_non_null/default 1.00 261.3±1.05µs 660.1
MB/sec 1.00 260.9±2.12µs 661.1 MB/sec
primitive_non_null/parquet_2 1.00 262.1±1.87µs 658.3
MB/sec 1.15 301.1±2.85µs 572.9 MB/sec
primitive_non_null/zstd 1.02 399.8±4.15µs 431.6
MB/sec 1.00 393.3±2.69µs 438.6 MB/sec
primitive_non_null/zstd_parquet_2 1.00 398.6±5.22µs 432.8
MB/sec 1.30 518.4±4.04µs 332.8 MB/sec
string/bloom_filter 1.00 663.8±4.72µs 3.0
GB/sec 1.69 1121.8±5.38µs 1825.7 MB/sec
string/default 1.00 418.0±1.79µs 4.8
GB/sec 1.00 419.7±5.55µs 4.8 GB/sec
string/parquet_2 1.00 433.1±1.89µs 4.6
GB/sec 1.43 618.4±6.76µs 3.2 GB/sec
string/zstd 1.00 1194.9±6.74µs 1714.0
MB/sec 1.02 1212.9±28.17µs 1688.6 MB/sec
string/zstd_parquet_2 1.00 1204.0±17.90µs 1701.1
MB/sec 1.41 1694.4±9.14µs 1208.7 MB/sec
string_and_binary_view/bloom_filter 1.00 296.9±2.99µs 425.0
MB/sec 1.02 304.3±2.02µs 414.7 MB/sec
string_and_binary_view/default 1.00 187.7±0.66µs 672.5
MB/sec 1.05 196.6±3.12µs 641.8 MB/sec
string_and_binary_view/parquet_2 1.00 191.8±2.10µs 657.8
MB/sec 1.05 201.4±0.88µs 626.5 MB/sec
string_and_binary_view/zstd 1.00 333.5±1.86µs 378.4
MB/sec 1.05 348.6±1.60µs 362.0 MB/sec
string_and_binary_view/zstd_parquet_2 1.00 326.6±2.85µs 386.4
MB/sec 1.07 349.3±1.93µs 361.3 MB/sec
string_dictionary/bloom_filter 1.00 331.9±2.10µs 3.0
GB/sec 1.00 330.8±3.34µs 3.0 GB/sec
string_dictionary/default 1.02 211.2±3.43µs 4.8
GB/sec 1.00 208.0±0.88µs 4.8 GB/sec
string_dictionary/parquet_2 1.01 211.3±2.05µs 4.8
GB/sec 1.00 209.0±1.22µs 4.8 GB/sec
string_dictionary/zstd 1.00 602.9±6.44µs 1711.7
MB/sec 1.00 601.1±3.78µs 1716.9 MB/sec
string_dictionary/zstd_parquet_2 1.00 597.6±3.65µs 1726.8
MB/sec 1.48 881.7±11.13µs 1170.5 MB/sec
string_non_null/bloom_filter 1.00 902.2±24.38µs 2.2
GB/sec 1.58 1424.6±21.18µs 1436.9 MB/sec
string_non_null/default 1.00 587.5±6.36µs 3.4
GB/sec 1.01 591.1±3.54µs 3.4 GB/sec
string_non_null/parquet_2 1.00 584.6±3.27µs 3.4
GB/sec 1.48 865.1±18.32µs 2.3 GB/sec
string_non_null/zstd 1.00 1634.6±15.53µs 1252.4
MB/sec 1.31 2.1±0.01ms 956.0 MB/sec
string_non_null/zstd_parquet_2 1.00 1630.4±7.24µs 1255.6
MB/sec 1.39 2.3±0.01ms 902.4 MB/sec
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
