adriangb commented on PR #11223: URL: https://github.com/apache/arrow-rs/pull/11223#issuecomment-5879237874
I am closing this PR. The work continues as a stack of smaller PRs, tracked in https://github.com/apache/arrow-rs/issues/11234: 1. https://github.com/apache/arrow-rs/pull/11235: release the bytes of a row group when the decoder is done with it 2. https://github.com/apache/arrow-rs/pull/11238: decode a batch as soon as its pages are pushed 3. https://github.com/apache/arrow-rs/pull/11239: release each page after the decoder has passed it 4. https://github.com/apache/arrow-rs/pull/11240: use the predicate cache in batch mode 5. https://github.com/apache/arrow-rs/pull/11241: use the configured selection policy in batch windows Two parts of this PR are not in the stack yet: `PushBuffers` kept sorted with binary-search lookups, and the release of the bytes of skipped row groups when a decoder is rebuilt. The branch `claude/push-decoder-batch-granular-on-11218` keeps them. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
