zhuqi-lucas opened a new pull request, #25686: URL: https://github.com/apache/datafusion/pull/25686
**Do not merge.** This PR exists only to run the DataFusion benchmarks against apache/arrow-rs#11168 (defer dictionary page decode on the Parquet skip path) before that PR is marked ready for review. ## What it does Pins every arrow-rs 60.0.0 crate in the workspace to the `lazy-dict-skip` branch (rev `f18aaab`) via `[patch.crates-io]`. No DataFusion code changes. The branch is arrow-rs `main` plus #11168, at the same 60.0.0 version DataFusion already depends on, so the diff against `main` is exactly the parquet change. ## Why an end-to-end run The arrow-rs side already ran `arrow_reader_clickbench` on CI: Q20 and Q21 (selective `URL LIKE` filters, so heavily skip-driven) were 20 to 23 percent faster, everything else within noise. That benchmark drives the Parquet reader directly. This run checks the same change through DataFusion's own scan path, including `pushdown_filters` and the predicate cache, which is where the skip-heavy access pattern actually comes from in a real engine. The change only affects column chunks that a scan skips into before it reads them: whole-chunk skips no longer decompress the dictionary at all, partial skips pay it on the first data page that needs it, and a plain sequential read is untouched. ## Tracking - arrow-rs PR: apache/arrow-rs#11168 - arrow-rs issue: apache/arrow-rs#11154 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
