giladkl opened a new pull request, #10689: URL: https://github.com/apache/arrow-rs/pull/10689
# Which issue does this PR close? - Closes #10688 # Rationale for this change col = 'x' / col <> '' over a Utf8View or BinaryView column is one of the hottest kernels in an analytical scan, and today it goes through the generic ArrayOrd path: each row is reached through a closure over an (array, index) pair, and settling it walks the full 128-bit view through a sequence of branches. For a short constant almost none of that is needed. A constant of four bytes or fewer is described entirely by a view's low 64 bits, which hold the length and the first four bytes, so masking those and comparing them against the constant resolved once up front settles a row with a single narrow integer compare over the flat &[u128] view slice. That loop is branch-free and vectorizes. # What changes are included in this PR? - arrow-ord: an eq_inline_scalar helper, and a guard in compare_op that routes to it for Op::Equal / Op::NotEqual when one side is a scalar. Everything else falls through to the existing generic path untouched: dictionary and REE inputs, a null scalar (which the fast path's null handling cannot express), non-view types, and constants longer than four bytes. - arrow: a stringview_scalar_eq benchmark group in comparison_kernels.rs sweeping three sizes, so the cache-resident and bandwidth-bound ends of the range are both visible. Constants wider than four bytes are deliberately left alone. They need the whole 128-bit view, and comparing that measured slower than the generic path's early exit on a length mismatch (a six-byte constant regressed 11.8%), so the cap is a measured limit rather than an arbitrary one. # Are these changes tested? Yes. Three tests are added alongside the existing byte-view comparison tests: - test_byte_view_eq_null_scalar — a null constant makes every row null, covering the shape the fast path declines. - test_byte_view_eq_null_row — null rows in the values array stay null. - test_byte_view_eq_scalar_either_side — the scalar on the left gives the same answer as on the right. The existing arrow-ord suite (272 tests) passes unchanged. Benchmark evidence, cargo bench --bench comparison_kernels -- stringview_scalar_eq on an Intel Xeon @ 2.80GHz (66 MiB L3): ┌───────────┬─────────┬─────────┬─────────┬────────┐ │ rows │ views │ before │ after │ change │ ├───────────┼─────────┼─────────┼─────────┼────────┤ │ 65,536 │ 1 MiB │ 111.4us │ 47.7us │ -57.2% │ ├───────────┼─────────┼─────────┼─────────┼────────┤ │ 1,048,576 │ 16 MiB │ 2.04ms │ 983.2us │ -51.7% │ ├───────────┼─────────┼─────────┼─────────┼────────┤ │ 8,388,608 │ 128 MiB │ 19.71ms │ 14.85ms │ -24.7% │ └───────────┴─────────┴─────────┴─────────┴────────┘ The win narrows at the largest size, where the views no longer fit in cache and the kernel becomes bound by memory bandwidth rather than by the comparison. # Are there any user-facing changes? Nope :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
