alamb commented on PR #16711: URL: https://github.com/apache/datafusion/pull/16711#issuecomment-3052818844
> One thing that I think has caused us problems is judging any improvements to pushdown based on not regressing performance when pushdown is enabled vs not. > > However, this makes making incremental progress really hard. What I think we should start doing is compare any proposed improvements to pushdown when pushdown is already on. > > In other words, let's make a benchmark that already has filter pushdown on. > > I'll make a PR for this new benchmark later today I actually found a seemingly good one here: https://github.com/apache/datafusion/blob/3ca09a642dac266dfdbf7f57d2a5af82a9c77436/benchmarks/bench.sh#L117-L116 ```shell bench.sh run parquet ``` I started it running and will see what happens. I need to do some other non parquet stuff for a few hours. Will be back -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
