tustvold commented on PR #2278:
URL: https://github.com/apache/arrow-rs/pull/2278#issuecomment-1202985899
Roughly ~10% performance improvement, which isn't bad given half the
objective was cleaning up the code with an eventual view to implementing #2257.
The major return for DeltaBitPackDecoder would likely only be realizable with
#2282 and larger miniblock sizes.
Performance vs master
```
arrow_array_reader/Int32Array/plain encoded, mandatory, no NULLs
time: [4.8459 us 4.8589 us 4.8732 us]
change: [+0.3572% +0.8566% +1.3081%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
3 (3.00%) high mild
3 (3.00%) high severe
arrow_array_reader/Int32Array/plain encoded, optional, no NULLs
time: [22.337 us 22.347 us 22.358 us]
change: [-1.0612% -0.7802% -0.5283%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severe
arrow_array_reader/Int32Array/plain encoded, optional, half NULLs
time: [28.798 us 28.815 us 28.833 us]
change: [-13.045% -12.928% -12.803%] (p = 0.00 <
0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
3 (3.00%) high mild
4 (4.00%) high severe
arrow_array_reader/Int32Array/binary packed, mandatory, no NULLs
time: [33.001 us 33.015 us 33.031 us]
change: [-18.056% -18.023% -17.989%] (p = 0.00 <
0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
4 (4.00%) high mild
4 (4.00%) high severe
arrow_array_reader/Int32Array/binary packed, optional, no NULLs
time: [50.711 us 50.729 us 50.747 us]
change: [-12.294% -12.252% -12.205%] (p = 0.00 <
0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severe
arrow_array_reader/Int32Array/binary packed, optional, half NULLs
time: [43.718 us 43.753 us 43.792 us]
change: [-15.172% -15.098% -15.016%] (p = 0.00 <
0.05)
Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
3 (3.00%) high mild
3 (3.00%) high severe
arrow_array_reader/Int32Array/dictionary encoded, mandatory, no NULLs
time: [33.403 us 33.410 us 33.419 us]
change: [-7.3073% -7.2057% -7.0230%] (p = 0.00 <
0.05)
Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
1 (1.00%) low severe
5 (5.00%) low mild
1 (1.00%) high mild
3 (3.00%) high severe
arrow_array_reader/Int32Array/dictionary encoded, optional, no NULLs
time: [51.100 us 51.116 us 51.133 us]
change: [-4.6051% -4.5621% -4.5166%] (p = 0.00 <
0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
6 (6.00%) high mild
1 (1.00%) high severe
arrow_array_reader/Int32Array/dictionary encoded, optional, half NULLs
time: [44.215 us 44.235 us 44.255 us]
change: [-9.9963% -9.9351% -9.8587%] (p = 0.00 <
0.05)
Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
3 (3.00%) high mild
3 (3.00%) high severe
arrow_array_reader/Int64Array/plain encoded, mandatory, no NULLs
time: [8.0967 us 8.1230 us 8.1534 us]
change: [+0.7474% +1.1695% +1.5384%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 9 outliers among 100 measurements (9.00%)
3 (3.00%) high mild
6 (6.00%) high severe
arrow_array_reader/Int64Array/plain encoded, optional, no NULLs
time: [25.602 us 25.631 us 25.665 us]
change: [-1.4309% -1.0332% -0.6523%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 7 outliers among 100 measurements (7.00%)
3 (3.00%) high mild
4 (4.00%) high severe
arrow_array_reader/Int64Array/plain encoded, optional, half NULLs
time: [31.643 us 31.672 us 31.700 us]
change: [+14.966% +15.048% +15.132%] (p = 0.00 <
0.05)
Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
10 (10.00%) high mild
arrow_array_reader/Int64Array/binary packed, mandatory, no NULLs
time: [62.860 us 62.874 us 62.891 us]
change: [-1.3340% -1.2969% -1.2569%] (p = 0.00 <
0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
6 (6.00%) high mild
2 (2.00%) high severe
arrow_array_reader/Int64Array/binary packed, optional, no NULLs
time: [80.933 us 80.982 us 81.059 us]
change: [-0.8099% -0.7311% -0.6512%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 9 outliers among 100 measurements (9.00%)
4 (4.00%) high mild
5 (5.00%) high severe
arrow_array_reader/Int64Array/binary packed, optional, half NULLs
time: [58.719 us 58.774 us 58.832 us]
change: [+6.0864% +6.1605% +6.2268%] (p = 0.00 <
0.05)
Performance has regressed.
Found 11 outliers among 100 measurements (11.00%)
7 (7.00%) high mild
4 (4.00%) high severe
arrow_array_reader/Int64Array/dictionary encoded, mandatory, no NULLs
time: [35.035 us 35.049 us 35.065 us]
change: [-12.831% -12.790% -12.750%] (p = 0.00 <
0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
3 (3.00%) high mild
5 (5.00%) high severe
arrow_array_reader/Int64Array/dictionary encoded, optional, no NULLs
time: [52.687 us 52.703 us 52.720 us]
change: [-8.9727% -8.9393% -8.9011%] (p = 0.00 <
0.05)
Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
5 (5.00%) high mild
1 (1.00%) high severe
arrow_array_reader/Int64Array/dictionary encoded, optional, half NULLs
time: [46.217 us 46.235 us 46.255 us]
change: [+4.6787% +4.7709% +4.8563%] (p = 0.00 <
0.05)
Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
1 (1.00%) low mild
4 (4.00%) high mild
2 (2.00%) high severe
arrow_array_reader/StringArray/plain encoded, mandatory, no NULLs
time: [175.32 us 175.44 us 175.56 us]
change: [-3.4189% -3.2578% -3.0783%] (p = 0.00 <
0.05)
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
arrow_array_reader/StringArray/plain encoded, optional, no NULLs
time: [194.47 us 194.59 us 194.74 us]
change: [-3.1158% -3.0118% -2.8869%] (p = 0.00 <
0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe
arrow_array_reader/StringArray/plain encoded, optional, half NULLs
time: [205.21 us 205.27 us 205.34 us]
change: [-6.1702% -6.0441% -5.9480%] (p = 0.00 <
0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
2 (2.00%) high mild
6 (6.00%) high severe
arrow_array_reader/StringArray/dictionary encoded, mandatory, no NULLs
time: [133.67 us 133.89 us 134.13 us]
change: [-4.3816% -4.2028% -4.0310%] (p = 0.00 <
0.05)
Performance has improved.
arrow_array_reader/StringArray/dictionary encoded, optional, no NULLs
time: [153.30 us 153.54 us 153.79 us]
change: [-1.8329% -1.6595% -1.5029%] (p = 0.00 <
0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
arrow_array_reader/StringArray/dictionary encoded, optional, half NULLs
time: [183.34 us 183.49 us 183.67 us]
change: [-6.1044% -6.0061% -5.8991%] (p = 0.00 <
0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high mild
arrow_array_reader/StringDictionary/dictionary encoded, mandatory, no NULLs
time: [26.463 us 26.471 us 26.480 us]
change: [-0.4894% -0.4345% -0.3755%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 7 outliers among 100 measurements (7.00%)
4 (4.00%) high mild
3 (3.00%) high severe
arrow_array_reader/StringDictionary/dictionary encoded, optional, no NULLs
time: [43.910 us 43.924 us 43.941 us]
change: [-0.0280% +0.0426% +0.1125%] (p = 0.24 >
0.05)
No change in performance detected.
Found 6 outliers among 100 measurements (6.00%)
3 (3.00%) high mild
3 (3.00%) high severe
arrow_array_reader/StringDictionary/dictionary encoded, optional, half NULLs
time: [49.736 us 49.766 us 49.803 us]
change: [+3.9807% +4.0557% +4.1406%] (p = 0.00 <
0.05)
Performance has regressed.
Found 9 outliers among 100 measurements (9.00%)
5 (5.00%) high mild
4 (4.00%) high severe
```
For completeness, performance vs master with `-C target-cpu=native`. So only
~5% but not a regression which is the important thing.
```
arrow_array_reader/Int32Array/plain encoded, mandatory, no NULLs
time: [4.7553 us 4.7598 us 4.7661 us]
change: [-0.0821% +0.7986% +1.5839%] (p = 0.06 >
0.05)
No change in performance detected.
Found 8 outliers among 100 measurements (8.00%)
2 (2.00%) high mild
6 (6.00%) high severe
arrow_array_reader/Int32Array/plain encoded, optional, no NULLs
time: [21.556 us 21.620 us 21.690 us]
change: [-0.9754% -0.5451% -0.0853%] (p = 0.02 <
0.05)
Change within noise threshold.
Found 13 outliers among 100 measurements (13.00%)
5 (5.00%) high mild
8 (8.00%) high severe
arrow_array_reader/Int32Array/plain encoded, optional, half NULLs
time: [28.930 us 28.942 us 28.957 us]
change: [-12.993% -12.935% -12.879%] (p = 0.00 <
0.05)
Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
6 (6.00%) high mild
4 (4.00%) high severe
arrow_array_reader/Int32Array/binary packed, mandatory, no NULLs
time: [30.420 us 30.429 us 30.440 us]
change: [+2.5445% +2.7910% +2.9347%] (p = 0.00 <
0.05)
Performance has regressed.
Found 15 outliers among 100 measurements (15.00%)
5 (5.00%) high mild
10 (10.00%) high severe
arrow_array_reader/Int32Array/binary packed, optional, no NULLs
time: [47.153 us 47.164 us 47.175 us]
change: [+1.0496% +1.4538% +1.8614%] (p = 0.00 <
0.05)
Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
1 (1.00%) low mild
3 (3.00%) high mild
6 (6.00%) high severe
arrow_array_reader/Int32Array/binary packed, optional, half NULLs
time: [42.822 us 42.856 us 42.897 us]
change: [-7.1747% -6.9019% -6.5972%] (p = 0.00 <
0.05)
Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
6 (6.00%) high mild
4 (4.00%) high severe
arrow_array_reader/Int32Array/dictionary encoded, mandatory, no NULLs
time: [30.892 us 30.899 us 30.905 us]
change: [-9.6399% -9.4767% -9.2311%] (p = 0.00 <
0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) high mild
2 (2.00%) high severe
arrow_array_reader/Int32Array/dictionary encoded, optional, no NULLs
time: [47.705 us 47.719 us 47.733 us]
change: [-6.2662% -6.0597% -5.9385%] (p = 0.00 <
0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) high mild
1 (1.00%) high severe
arrow_array_reader/Int32Array/dictionary encoded, optional, half NULLs
time: [43.416 us 43.432 us 43.449 us]
change: [-9.5642% -9.3380% -9.2034%] (p = 0.00 <
0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe
arrow_array_reader/Int64Array/plain encoded, mandatory, no NULLs
time: [7.9014 us 7.9765 us 8.0715 us]
change: [-2.1869% -1.2440% -0.1997%] (p = 0.01 <
0.05)
Change within noise threshold.
Found 17 outliers among 100 measurements (17.00%)
17 (17.00%) high severe
arrow_array_reader/Int64Array/plain encoded, optional, no NULLs
time: [25.072 us 25.171 us 25.303 us]
change: [-3.4804% -2.0406% -0.7844%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 14 outliers among 100 measurements (14.00%)
3 (3.00%) high mild
11 (11.00%) high severe
arrow_array_reader/Int64Array/plain encoded, optional, half NULLs
time: [30.868 us 30.886 us 30.907 us]
change: [-10.822% -10.526% -10.232%] (p = 0.00 <
0.05)
Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
5 (5.00%) high mild
1 (1.00%) high severe
arrow_array_reader/Int64Array/binary packed, mandatory, no NULLs
time: [62.747 us 62.765 us 62.787 us]
change: [+0.1408% +0.4562% +0.7639%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 7 outliers among 100 measurements (7.00%)
3 (3.00%) high mild
4 (4.00%) high severe
arrow_array_reader/Int64Array/binary packed, optional, no NULLs
time: [79.580 us 79.602 us 79.628 us]
change: [+0.4857% +0.6106% +0.8242%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 14 outliers among 100 measurements (14.00%)
1 (1.00%) low mild
9 (9.00%) high mild
4 (4.00%) high severe
arrow_array_reader/Int64Array/binary packed, optional, half NULLs
time: [57.406 us 57.415 us 57.424 us]
change: [-8.3594% -8.0473% -7.7786%] (p = 0.00 <
0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severe
arrow_array_reader/Int64Array/dictionary encoded, mandatory, no NULLs
time: [32.319 us 32.323 us 32.328 us]
change: [-9.0431% -9.0036% -8.9690%] (p = 0.00 <
0.05)
Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
6 (6.00%) high mild
arrow_array_reader/Int64Array/dictionary encoded, optional, no NULLs
time: [49.277 us 49.289 us 49.305 us]
change: [-5.9239% -5.6357% -5.3767%] (p = 0.00 <
0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
1 (1.00%) low mild
2 (2.00%) high mild
4 (4.00%) high severe
arrow_array_reader/Int64Array/dictionary encoded, optional, half NULLs
time: [44.122 us 44.141 us 44.163 us]
change: [-10.843% -10.591% -10.341%] (p = 0.00 <
0.05)
Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
5 (5.00%) high mild
4 (4.00%) high severe
arrow_array_reader/StringArray/plain encoded, mandatory, no NULLs
time: [176.52 us 176.59 us 176.65 us]
change: [-7.9276% -7.8375% -7.7034%] (p = 0.00 <
0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
2 (2.00%) low mild
3 (3.00%) high mild
3 (3.00%) high severe
arrow_array_reader/StringArray/plain encoded, optional, no NULLs
time: [194.73 us 194.83 us 194.94 us]
change: [-8.9202% -8.6621% -8.4009%] (p = 0.00 <
0.05)
Performance has improved.
Found 15 outliers among 100 measurements (15.00%)
12 (12.00%) low mild
2 (2.00%) high mild
1 (1.00%) high severe
arrow_array_reader/StringArray/plain encoded, optional, half NULLs
time: [218.35 us 218.44 us 218.55 us]
change: [-3.4216% -3.1171% -2.7915%] (p = 0.00 <
0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
1 (1.00%) low mild
2 (2.00%) high mild
4 (4.00%) high severe
arrow_array_reader/StringArray/dictionary encoded, mandatory, no NULLs
time: [135.36 us 135.61 us 135.90 us]
change: [-2.4626% -1.9884% -1.5822%] (p = 0.00 <
0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe
arrow_array_reader/StringArray/dictionary encoded, optional, no NULLs
time: [153.32 us 153.48 us 153.67 us]
change: [-1.2992% -1.0836% -0.7954%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
6 (6.00%) high mild
2 (2.00%) high severe
arrow_array_reader/StringArray/dictionary encoded, optional, half NULLs
time: [197.32 us 197.47 us 197.64 us]
change: [+1.4295% +1.6590% +1.8209%] (p = 0.00 <
0.05)
Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) high mild
1 (1.00%) high severe
arrow_array_reader/StringDictionary/dictionary encoded, mandatory, no NULLs
time: [25.246 us 25.256 us 25.265 us]
change: [-3.9833% -3.8835% -3.7605%] (p = 0.00 <
0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
4 (4.00%) high mild
3 (3.00%) high severe
arrow_array_reader/StringDictionary/dictionary encoded, optional, no NULLs
time: [42.021 us 42.028 us 42.037 us]
change: [-2.7292% -2.4821% -2.3167%] (p = 0.00 <
0.05)
Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
4 (4.00%) high mild
5 (5.00%) high severe
arrow_array_reader/StringDictionary/dictionary encoded, optional, half NULLs
time: [48.923 us 48.940 us 48.959 us]
change: [+4.7617% +5.0899% +5.3886%] (p = 0.00 <
0.05)
Performance has regressed.
Found 9 outliers among 100 measurements (9.00%)
1 (1.00%) low mild
4 (4.00%) high mild
4 (4.00%) high severe
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
