alamb commented on PR #10136: URL: https://github.com/apache/arrow-rs/pull/10136#issuecomment-5830824281
I merged up from main to resolve a merge conflict. I also re-ran this code on ``` ==> Host CPU Vendor ID: GenuineIntel Model name: Intel(R) Xeon(R) CPU @ 3.10GHz ==> Target features enabled by -C target-cpu=native target_feature="adx" target_feature="aes" target_feature="avx" target_feature="avx2" target_feature="avx512bw" target_feature="avx512cd" target_feature="avx512dq" target_feature="avx512f" target_feature="avx512vl" target_feature="avx512vnni" target_feature="bmi1" target_feature="bmi2" target_feature="cmpxchg16b" target_feature="f16c" target_feature="fma" target_feature="fxsr" target_feature="lzcnt" target_feature="movbe" target_feature="pclmulqdq" target_feature="popcnt" target_feature="rdrand" target_feature="rdseed" target_feature="sse" target_feature="sse2" target_feature="sse3" target_feature="sse4.1" target_feature="sse4.2" target_feature="ssse3" target_feature="xsave" target_feature="xsavec" target_feature="xsaveopt" target_feature="xsaves" bmi2: ENABLED (pext path will be compiled) ``` There is quite a bit of noise, but I think overall this looks like a win to me <details> ``` ==> Paired runs, only rows differing by more than 5% --- base-1 vs pr-1 group base-1 pr-1 ----- ------ ---- filter context decimal128 (kept 1/2) 1.00 46.4±0.92µs ? ?/sec 1.39 64.4±0.45µs ? ?/sec filter context decimal128 high selectivity (kept 1023/1024) 1.00 48.9±1.40µs ? ?/sec 1.12 54.8±3.52µs ? ?/sec filter context f32 (kept 1/2) 1.77 117.8±0.19µs ? ?/sec 1.00 66.7±0.65µs ? ?/sec filter context fsb with value length 20 (kept 1/2) 1.00 73.1±3.50µs ? ?/sec 1.19 86.9±2.59µs ? ?/sec filter context fsb with value length 50 (kept 1/2) 1.00 163.0±12.11µs ? ?/sec 1.19 194.0±3.51µs ? ?/sec filter context fsb with value length 50 high selectivity (kept 1023/1024) 1.00 187.4±3.81µs ? ?/sec 1.28 239.0±0.97µs ? ?/sec filter context fsb with value length 50 low selectivity (kept 1/1024) 1.14 341.1±4.34ns ? ?/sec 1.00 298.6±2.94ns ? ?/sec filter context i32 high selectivity (kept 1023/1024) 1.17 7.4±0.29µs ? ?/sec 1.00 6.3±0.23µs ? ?/sec filter context i32 w NULLs (kept 1/2) 1.77 118.2±1.12µs ? ?/sec 1.00 66.7±0.07µs ? ?/sec filter context i32 w NULLs high selectivity (kept 1023/1024) 1.06 8.6±0.36µs ? ?/sec 1.00 8.1±0.03µs ? ?/sec filter context mixed string view (kept 1/2) 1.72 104.7±1.01µs ? ?/sec 1.00 60.9±4.73µs ? ?/sec filter context mixed string view high selectivity (kept 1023/1024) 1.00 54.0±0.91µs ? ?/sec 1.19 64.3±0.18µs ? ?/sec filter context short string view (kept 1/2) 1.73 109.5±2.18µs ? ?/sec 1.00 63.3±5.50µs ? ?/sec filter context string (kept 1/2) 1.14 563.7±10.80µs ? ?/sec 1.00 496.5±2.11µs ? ?/sec filter context string dictionary w NULLs (kept 1/2) 1.77 118.3±1.44µs ? ?/sec 1.00 66.9±0.68µs ? ?/sec filter context string dictionary w NULLs high selectivity (kept 1023/1024) 1.00 8.3±0.28µs ? ?/sec 1.07 8.9±0.26µs ? ?/sec filter context u8 high selectivity (kept 1023/1024) 1.00 1727.5±31.04ns ? ?/sec 1.06 1824.7±10.44ns ? ?/sec filter context u8 w NULLs (kept 1/2) 4.44 66.2±1.23µs ? ?/sec 1.00 14.9±0.15µs ? ?/sec filter decimal128 (kept 1/2) 1.00 39.2±4.29µs ? ?/sec 1.20 47.2±6.34µs ? ?/sec filter decimal128 high selectivity (kept 1023/1024) 1.00 51.2±1.46µs ? ?/sec 1.16 59.5±5.40µs ? ?/sec filter f32 (kept 1/2) 3.61 124.1±0.42µs ? ?/sec 1.00 34.4±0.31µs ? ?/sec filter fsb with value length 20 high selectivity (kept 1023/1024) 1.00 70.1±2.07µs ? ?/sec 1.08 75.7±3.48µs ? ?/sec filter fsb with value length 20 low selectivity (kept 1/1024) 1.00 2.2±0.02µs ? ?/sec 1.31 2.9±0.00µs ? ?/sec filter fsb with value length 5 low selectivity (kept 1/1024) 1.00 2.1±0.02µs ? ?/sec 1.29 2.7±0.03µs ? ?/sec filter fsb with value length 50 (kept 1/2) 1.00 140.6±6.68µs ? ?/sec 1.25 176.1±17.51µs ? ?/sec filter fsb with value length 50 high selectivity (kept 1023/1024) 1.00 198.6±15.64µs ? ?/sec 1.15 227.9±3.12µs ? ?/sec filter fsb with value length 50 low selectivity (kept 1/1024) 1.00 2.2±0.01µs ? ?/sec 1.31 2.8±0.01µs ? ?/sec --- base-2 vs pr-2 group base-2 pr-2 ----- ------ ---- filter context decimal128 (kept 1/2) 1.16 67.2±3.46µs ? ?/sec 1.00 57.7±5.57µs ? ?/sec filter context decimal128 high selectivity (kept 1023/1024) 1.00 49.9±0.62µs ? ?/sec 1.16 57.6±3.54µs ? ?/sec filter context decimal128 low selectivity (kept 1/1024) 1.09 189.7±3.57ns ? ?/sec 1.00 174.3±1.89ns ? ?/sec filter context f32 (kept 1/2) 1.77 118.0±1.10µs ? ?/sec 1.00 66.8±0.63µs ? ?/sec filter context fsb with value length 20 (kept 1/2) 1.16 84.8±3.18µs ? ?/sec 1.00 72.8±1.97µs ? ?/sec filter context fsb with value length 20 high selectivity (kept 1023/1024) 1.17 78.6±0.86µs ? ?/sec 1.00 66.9±2.79µs ? ?/sec filter context fsb with value length 20 low selectivity (kept 1/1024) 1.00 300.6±4.32ns ? ?/sec 1.11 333.1±3.88ns ? ?/sec filter context fsb with value length 5 high selectivity (kept 1023/1024) 1.25 9.9±0.22µs ? ?/sec 1.00 7.9±0.25µs ? ?/sec filter context fsb with value length 50 (kept 1/2) 1.29 195.4±3.61µs ? ?/sec 1.00 151.6±8.88µs ? ?/sec filter context fsb with value length 50 high selectivity (kept 1023/1024) 1.08 204.5±6.19µs ? ?/sec 1.00 189.3±6.46µs ? ?/sec filter context fsb with value length 50 low selectivity (kept 1/1024) 1.00 312.3±4.42ns ? ?/sec 1.07 332.8±3.96ns ? ?/sec filter context i32 high selectivity (kept 1023/1024) 1.00 5.9±0.17µs ? ?/sec 1.05 6.2±0.23µs ? ?/sec filter context i32 w NULLs (kept 1/2) 1.77 117.9±1.56µs ? ?/sec 1.00 66.8±0.09µs ? ?/sec filter context mixed string view (kept 1/2) 1.96 112.9±2.79µs ? ?/sec 1.00 57.6±2.08µs ? ?/sec filter context short string view (kept 1/2) 1.98 115.5±3.96µs ? ?/sec 1.00 58.3±0.83µs ? ?/sec filter context short string view high selectivity (kept 1023/1024) 1.22 63.9±3.03µs ? ?/sec 1.00 52.4±1.78µs ? ?/sec filter context string (kept 1/2) 1.14 545.3±7.43µs ? ?/sec 1.00 479.4±6.52µs ? ?/sec filter context string dictionary high selectivity (kept 1023/1024) 1.00 6.0±0.07µs ? ?/sec 1.06 6.4±0.25µs ? ?/sec filter context string dictionary w NULLs (kept 1/2) 1.77 117.9±0.32µs ? ?/sec 1.00 66.7±0.10µs ? ?/sec filter context string high selectivity (kept 1023/1024) 1.05 671.6±17.79µs ? ?/sec 1.00 638.8±9.68µs ? ?/sec filter context u8 w NULLs (kept 1/2) 4.41 65.9±0.60µs ? ?/sec 1.00 15.0±0.18µs ? ?/sec filter decimal128 (kept 1/2) 1.00 38.8±0.15µs ? ?/sec 1.14 44.2±5.61µs ? ?/sec filter decimal128 high selectivity (kept 1023/1024) 1.07 60.4±0.55µs ? ?/sec 1.00 56.2±0.56µs ? ?/sec filter f32 (kept 1/2) 3.61 123.9±0.36µs ? ?/sec 1.00 34.4±0.35µs ? ?/sec filter fsb with value length 20 low selectivity (kept 1/1024) 1.00 2.2±0.03µs ? ?/sec 1.30 2.8±0.03µs ? ?/sec filter fsb with value length 5 high selectivity (kept 1023/1024) 1.14 11.6±0.13µs ? ?/sec 1.00 10.2±0.54µs ? ?/sec filter fsb with value length 5 low selectivity (kept 1/1024) 1.00 2.1±0.03µs ? ?/sec 1.29 2.7±0.03µs ? ?/sec filter fsb with value length 50 (kept 1/2) 1.29 185.9±5.06µs ? ?/sec 1.00 144.1±10.29µs ? ?/sec filter fsb with value length 50 high selectivity (kept 1023/1024) 1.24 245.3±2.78µs ? ?/sec 1.00 197.3±7.03µs ? ?/sec filter fsb with value length 50 low selectivity (kept 1/1024) 1.00 2.1±0.03µs ? ?/sec 1.34 2.9±0.03µs ? ?/sec --- base-3 vs pr-3 group base-3 pr-3 ----- ------ ---- filter context decimal128 (kept 1/2) 1.00 54.3±0.54µs ? ?/sec 1.15 62.5±0.25µs ? ?/sec filter context f32 (kept 1/2) 1.76 117.8±0.16µs ? ?/sec 1.00 67.0±0.63µs ? ?/sec filter context fsb with value length 20 (kept 1/2) 1.08 86.7±1.17µs ? ?/sec 1.00 80.5±5.62µs ? ?/sec filter context fsb with value length 20 high selectivity (kept 1023/1024) 1.09 87.1±0.62µs ? ?/sec 1.00 79.7±0.38µs ? ?/sec filter context fsb with value length 5 high selectivity (kept 1023/1024) 1.00 8.6±0.18µs ? ?/sec 1.13 9.7±0.63µs ? ?/sec filter context fsb with value length 50 (kept 1/2) 1.10 193.2±4.92µs ? ?/sec 1.00 175.2±9.39µs ? ?/sec filter context fsb with value length 50 low selectivity (kept 1/1024) 1.00 295.1±3.44ns ? ?/sec 1.08 318.4±3.69ns ? ?/sec filter context i32 w NULLs (kept 1/2) 1.76 117.9±1.19µs ? ?/sec 1.00 67.0±0.78µs ? ?/sec filter context i32 w NULLs high selectivity (kept 1023/1024) 1.00 8.3±0.11µs ? ?/sec 1.09 9.1±0.43µs ? ?/sec filter context mixed string view (kept 1/2) 1.76 117.6±1.38µs ? ?/sec 1.00 67.0±3.74µs ? ?/sec filter context mixed string view high selectivity (kept 1023/1024) 1.06 57.8±1.52µs ? ?/sec 1.00 54.6±1.85µs ? ?/sec filter context short string view (kept 1/2) 1.81 120.5±0.29µs ? ?/sec 1.00 66.5±0.38µs ? ?/sec filter context short string view high selectivity (kept 1023/1024) 1.00 52.6±2.25µs ? ?/sec 1.26 66.4±0.28µs ? ?/sec filter context string (kept 1/2) 1.08 541.5±6.25µs ? ?/sec 1.00 503.3±4.90µs ? ?/sec filter context string dictionary w NULLs (kept 1/2) 1.76 118.1±1.16µs ? ?/sec 1.00 67.1±0.74µs ? ?/sec filter context u8 high selectivity (kept 1023/1024) 1.00 1714.0±17.42ns ? ?/sec 1.19 2.0±0.02µs ? ?/sec filter context u8 w NULLs (kept 1/2) 4.41 65.9±0.65µs ? ?/sec 1.00 14.9±0.05µs ? ?/sec filter context u8 w NULLs high selectivity (kept 1023/1024) 1.06 4.1±0.04µs ? ?/sec 1.00 3.9±0.05µs ? ?/sec filter decimal128 (kept 1/2) 1.00 39.5±0.32µs ? ?/sec 1.34 52.8±0.14µs ? ?/sec filter decimal128 high selectivity (kept 1023/1024) 1.00 54.1±3.46µs ? ?/sec 1.16 62.7±0.57µs ? ?/sec filter f32 (kept 1/2) 3.61 124.0±1.35µs ? ?/sec 1.00 34.4±0.41µs ? ?/sec filter fsb with value length 20 high selectivity (kept 1023/1024) 1.19 82.7±4.80µs ? ?/sec 1.00 69.6±1.47µs ? ?/sec filter fsb with value length 20 low selectivity (kept 1/1024) 1.00 2.2±0.02µs ? ?/sec 1.27 2.8±0.01µs ? ?/sec filter fsb with value length 5 high selectivity (kept 1023/1024) 1.00 10.7±0.28µs ? ?/sec 1.07 11.4±0.28µs ? ?/sec filter fsb with value length 5 low selectivity (kept 1/1024) 1.00 2.2±0.03µs ? ?/sec 1.26 2.7±0.00µs ? ?/sec filter fsb with value length 50 (kept 1/2) 1.06 183.6±1.38µs ? ?/sec 1.00 173.3±7.81µs ? ?/sec filter fsb with value length 50 high selectivity (kept 1023/1024) 1.20 265.4±1.94µs ? ?/sec 1.00 220.3±3.60µs ? ?/sec filter fsb with value length 50 low selectivity (kept 1/1024) 1.00 2.2±0.02µs ? ?/sec 1.32 2.8±0.01µs ? ?/sec --- base-4 vs pr-4 group base-4 pr-4 ----- ------ ---- filter context decimal128 (kept 1/2) 1.25 61.5±3.11µs ? ?/sec 1.00 49.3±0.86µs ? ?/sec filter context decimal128 high selectivity (kept 1023/1024) 1.00 56.1±0.57µs ? ?/sec 1.07 59.8±0.22µs ? ?/sec filter context f32 (kept 1/2) 1.76 118.0±0.31µs ? ?/sec 1.00 66.9±0.71µs ? ?/sec filter context f32 high selectivity (kept 1023/1024) 1.05 8.8±0.05µs ? ?/sec 1.00 8.3±0.07µs ? ?/sec filter context fsb with value length 20 high selectivity (kept 1023/1024) 1.20 86.6±8.46µs ? ?/sec 1.00 72.3±0.57µs ? ?/sec filter context fsb with value length 50 high selectivity (kept 1023/1024) 1.12 245.7±0.36µs ? ?/sec 1.00 220.2±3.03µs ? ?/sec filter context i32 w NULLs (kept 1/2) 1.77 117.9±1.12µs ? ?/sec 1.00 66.8±0.08µs ? ?/sec filter context i32 w NULLs high selectivity (kept 1023/1024) 1.07 8.6±0.16µs ? ?/sec 1.00 8.1±0.02µs ? ?/sec filter context mixed string view (kept 1/2) 1.71 117.9±0.88µs ? ?/sec 1.00 69.0±0.30µs ? ?/sec filter context short string view (kept 1/2) 1.84 113.8±3.78µs ? ?/sec 1.00 61.8±5.17µs ? ?/sec filter context string (kept 1/2) 1.10 560.3±5.19µs ? ?/sec 1.00 511.6±9.17µs ? ?/sec filter context string dictionary w NULLs (kept 1/2) 1.76 117.9±0.25µs ? ?/sec 1.00 67.0±0.70µs ? ?/sec filter context u8 high selectivity (kept 1023/1024) 1.16 2.0±0.02µs ? ?/sec 1.00 1737.1±19.60ns ? ?/sec filter context u8 w NULLs (kept 1/2) 4.40 65.8±0.10µs ? ?/sec 1.00 14.9±0.04µs ? ?/sec filter decimal128 high selectivity (kept 1023/1024) 1.26 65.2±0.34µs ? ?/sec 1.00 51.6±0.92µs ? ?/sec filter f32 (kept 1/2) 3.60 124.0±1.25µs ? ?/sec 1.00 34.4±0.38µs ? ?/sec filter fsb with value length 20 high selectivity (kept 1023/1024) 1.05 81.5±0.65µs ? ?/sec 1.00 77.4±0.45µs ? ?/sec filter fsb with value length 20 low selectivity (kept 1/1024) 1.00 2.2±0.03µs ? ?/sec 1.26 2.8±0.01µs ? ?/sec filter fsb with value length 5 low selectivity (kept 1/1024) 1.00 2.1±0.00µs ? ?/sec 1.29 2.7±0.00µs ? ?/sec filter fsb with value length 50 (kept 1/2) 1.00 164.7±12.08µs ? ?/sec 1.08 178.2±0.71µs ? ?/sec filter fsb with value length 50 high selectivity (kept 1023/1024) 1.09 264.7±1.74µs ? ?/sec 1.00 243.4±9.09µs ? ?/sec filter fsb with value length 50 low selectivity (kept 1/1024) 1.00 2.2±0.00µs ? ?/sec 1.29 2.8±0.03µs ? ?/sec filter i32 high selectivity (kept 1023/1024) 1.06 7.7±0.39µs ? ?/sec 1.00 7.3±0.08µs ? ?/sec --- base-5 vs pr-5 group base-5 pr-5 ----- ------ ---- filter context decimal128 high selectivity (kept 1023/1024) 1.09 54.9±3.98µs ? ?/sec 1.00 50.3±0.68µs ? ?/sec filter context f32 (kept 1/2) 1.76 117.9±1.14µs ? ?/sec 1.00 66.9±0.65µs ? ?/sec filter context f32 high selectivity (kept 1023/1024) 1.06 8.9±0.11µs ? ?/sec 1.00 8.4±0.35µs ? ?/sec filter context fsb with value length 5 high selectivity (kept 1023/1024) 1.37 11.6±0.18µs ? ?/sec 1.00 8.5±0.40µs ? ?/sec filter context fsb with value length 50 low selectivity (kept 1/1024) 1.00 287.1±3.03ns ? ?/sec 1.15 329.7±0.34ns ? ?/sec filter context i32 w NULLs (kept 1/2) 1.76 117.9±1.21µs ? ?/sec 1.00 66.8±0.29µs ? ?/sec filter context i32 w NULLs high selectivity (kept 1023/1024) 1.07 9.0±0.26µs ? ?/sec 1.00 8.4±0.16µs ? ?/sec filter context mixed string view (kept 1/2) 2.13 122.0±0.89µs ? ?/sec 1.00 57.3±0.84µs ? ?/sec filter context mixed string view high selectivity (kept 1023/1024) 1.05 55.5±1.05µs ? ?/sec 1.00 52.8±0.61µs ? ?/sec filter context short string view (kept 1/2) 1.97 121.8±0.87µs ? ?/sec 1.00 61.7±0.99µs ? ?/sec filter context short string view high selectivity (kept 1023/1024) 1.09 57.9±2.44µs ? ?/sec 1.00 53.2±0.16µs ? ?/sec filter context string (kept 1/2) 1.10 526.6±8.26µs ? ?/sec 1.00 479.2±3.04µs ? ?/sec filter context string dictionary w NULLs (kept 1/2) 1.76 118.2±0.92µs ? ?/sec 1.00 67.0±0.71µs ? ?/sec filter context u8 w NULLs (kept 1/2) 4.42 65.9±0.60µs ? ?/sec 1.00 14.9±0.10µs ? ?/sec filter context u8 w NULLs high selectivity (kept 1023/1024) 1.07 4.3±0.04µs ? ?/sec 1.00 4.0±0.08µs ? ?/sec filter decimal128 high selectivity (kept 1023/1024) 1.16 61.9±0.19µs ? ?/sec 1.00 53.4±0.28µs ? ?/sec filter f32 (kept 1/2) 3.61 124.0±0.79µs ? ?/sec 1.00 34.3±0.12µs ? ?/sec filter fsb with value length 20 low selectivity (kept 1/1024) 1.00 2.2±0.03µs ? ?/sec 1.25 2.8±0.01µs ? ?/sec filter fsb with value length 5 high selectivity (kept 1023/1024) 1.27 13.2±0.24µs ? ?/sec 1.00 10.4±0.41µs ? ?/sec filter fsb with value length 5 low selectivity (kept 1/1024) 1.00 2.2±0.03µs ? ?/sec 1.25 2.8±0.00µs ? ?/sec filter fsb with value length 50 high selectivity (kept 1023/1024) 1.00 194.7±9.60µs ? ?/sec 1.13 220.7±15.98µs ? ?/sec filter fsb with value length 50 low selectivity (kept 1/1024) 1.00 2.2±0.03µs ? ?/sec 1.30 2.8±0.03µs ? ?/sec ``` </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
