XiangpengHao commented on PR #7850: URL: https://github.com/apache/arrow-rs/pull/7850#issuecomment-3082355027
> Summary so far (I now need to go work on some other things for the rest of the day): > > I made two proposed changes > > * [Simplify projection caching XiangpengHao/arrow-rs#5](https://github.com/XiangpengHao/arrow-rs/pull/5) > * [Move cache options construction to ArrayReaderBuilder, add builders XiangpengHao/arrow-rs#6](https://github.com/XiangpengHao/arrow-rs/pull/6) > * I found a bug in this code via the DataFusion code (see [POC: Test DataFusion with experimental Parquet Filter Pushdown (try 4) datafusion#16711 (comment)](https://github.com/apache/datafusion/pull/16711#issuecomment-3078620431z)) > > My plan for tomorrow will be to try and write some tests: > > 1. Reproduce the bug / error in an arrow-rs only test > 2. Write some sort of integration test that shows the cache working (in preparation for wiring in the memory limit) Thank you for the review @alamb , I plan to take a look on this in the next few days, and also think about further optimizations. Maybe it's just me, but I can't reproduce some of the regressions reported in datafusion integrations, I'll get a "cloud" machine and try again. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
