alamb commented on PR #7850: URL: https://github.com/apache/arrow-rs/pull/7850#issuecomment-3114685204
Update: Here is a proposed PR to add a config option for predicate evaluation: - https://github.com/XiangpengHao/arrow-rs/pull/7 The only thing left is to figure out how to integrate this caching into the sync reader. Some options I can think of: 1. Don't integrate it at all 2. Rework the IO pattern in the sync reader so it only evaluates predicates a row group at a time rather than for the entire file (which is likely hard due to) https://github.com/apache/arrow-rs/issues/7983 So now I am stumped 🤔 Maybe I'll try and hack out a ParquetDecoder and see what that would look like 🤔 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
