Rich-T-kid opened a new pull request, #11119: URL: https://github.com/apache/arrow-rs/pull/11119
## What Adds `InProgressFixedSizeBinaryArray`, a specialized `InProgressArray` for `FixedSizeBinary(N)` columns in `BatchCoalescer`. ## Why Previously, `FixedSizeBinary` went through `GenericInProgressArray` → `concat_fallback` → `MutableArrayData`, which has boxed-closure dispatch overhead and rebuilds the array through a generic infrastructure. `FixedSizeBinaryBuilder::append_array` already does a direct `extend_from_slice` on the value buffer and a single `append_buffer` / `append_n_non_nulls` on the null bitmap — this PR wires it up as an incremental coalesce path with pre-allocated buffer capacity. ## Benchmarks Run with `cargo bench --bench coalesce_kernels --features test_utils -- fsb`. Benchmark environment: Apple M-series, batch_size=8192, comparing generic path (before) vs specialized path (after). **Filter benchmarks, nulls=0:** | benchmark | before | after | change | |-----------|--------|-------|--------| | fsb16 sel=0.1% | 27.1 ms | 18.7 ms | **-31%** | | fsb32 sel=0.1% | 28.2 ms | 18.9 ms | **-33%** | | fsb16 sel=1% | 3.5 ms | 3.0 ms | **-13%** | | fsb32 sel=1% | 4.1 ms | 3.3 ms | **-19%** | | fsb16 sel=10% | 1.84 ms | 1.85 ms | ~0% | | fsb32 sel=10% | 2.19 ms | 2.19 ms | ~0% | | fsb16 sel=80% | 570 µs | 564 µs | ~0% | | fsb32 sel=80% | ~920 µs | ~920 µs | ~0% | The largest gains are at low selectivity — exactly the regime where `BatchCoalescer` is most commonly used (after a filter that keeps few rows). At high selectivity the `copy_rows_by_filter_from` path calls `filter.filter()` to materialize the selection first, then `append_array`; both paths have the same asymptotic behavior there. Part of https://github.com/apache/arrow-rs/issues/7761 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
