cakeni opened a new pull request, #11655: URL: https://github.com/apache/arrow-rs/pull/11655
# Which issue does this PR close? Closes #11635. # Rationale for this change The sparse `push_batch_with_filter` path retains every input data buffer for a few selected StringView or BinaryView values. Materializing the filter and calling `push_batch` already compacts these buffers. # What changes are included in this PR? Pass the filtered views through `set_source` and `copy_rows`, then clear the temporary source. This reuses the existing compaction threshold, buffer copying and memory accounting. The inline fast path is retained, and sufficiently dense filtered layouts still share their source buffers. # Are these changes tested? A regression compares values and retained buffer capacity with `filter_record_batch` plus `push_batch` for StringView and BinaryView, including nulls, inline values, slices and multiple source buffers. Separate checks cover dropping unused external buffers and retaining sharing for dense data. For the reported shape (1,600 values of 39 bytes, selecting every 100th row), retained data-buffer capacity drops from 114,688 to 8,192 bytes for both StringView and BinaryView. This measures the output's data buffers, excluding views and validity buffers. # Are there any significant user-facing changes? No public API changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
