alamb opened a new pull request #657: URL: https://github.com/apache/arrow-datafusion/pull/657
This is the test harness I intend to use to validate the fix for https://github.com/apache/arrow-datafusion/issues/656 and https://github.com/apache/arrow-datafusion/issues/649. # Rationale: Parquet pruning is broken (https://github.com/apache/arrow-datafusion/issues/656 see https://github.com/apache/arrow-datafusion/issues/649) but all our tests are passing. This is not good ... The test and infrastructure is pretty large itself so I wanted to get it reviewed separately prior to the actual bug fixes # Changes 1. Add an end-to-end test for parquet pruning 2. Add statistics to parquet reader (that are used in the tests) 3. Add a "plan_statistics" function to gather all `SQLMetrics` after a plan has been executed I plan to extend the parquet test to cover more cases as I work on fixing the bugs # Are there any user-facing changes? There are some new user facing SQL metrics -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
