neilconway opened a new pull request, #11270: URL: https://github.com/apache/arrow-rs/pull/11270
# Which issue does this PR close? - N/A # Rationale for this change `concat_elements_bytes` and `concat_elements_utf8_many` sized their output capacities using the size of the underlying input values array minus the first offset. This can result in over-allocation when the input is sliced to omit values at the suffix of the values array. Benchmarks: - Two inputs, compact: 48.79 → 48.94 µs (0.31% slower). Allocated output value-buffer capacity is 469,824 bytes (unchanged). - Three inputs, compact: 87.73 → 84.91 µs (3.21% less time). Output capacity is 704,736 bytes (unchanged). - Two inputs, sliced: 49.24 → 49.38 µs (0.27% slower). Capacity 7,078,784 → 469,824 bytes, a 93.36% reduction. - Three inputs, sliced: 88.69 → 85.18 µs (3.96% less time). Capacity 10,618,176 → 704,736 bytes, a 93.36% reduction. # What changes are included in this PR? * Fix output buffer sizing for `concat_elements_bytes` and `concat_elements_utf8_many` for sliced inputs * Add benchmark * Add unit test # Are these changes tested? Yes; existing tests pass, new test added. # Are there any user-facing changes? No. # AI usage Developed with Codex (Astra 6), reviewed and revised with Claude Code (Opus 5.5). I have reviewed and understand the resulting code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
