Rich-T-kid commented on PR #26789: URL: https://github.com/apache/datafusion/pull/26789#issuecomment-6857256535
### Current `main` vs PR head, with RLE dictionary reads disabled This is the requested merge-candidate comparison: - baseline: `main` at `4c7b30752c159a85c713453ad7041b9c29e4d647` - changed: this PR at `678d97de914fd0e55fbc1f2d5eea6ac67a916dff` - both sides: `DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY=false` - changed side only: `DATAFUSION_OPTIMIZER_ENABLE_DICTIONARY_AGGREGATION=true` | Suite | Minimum total | Mean total | Raw slower flags | |---|---:|---:|---| | [TPC-H](https://github.com/apache/datafusion/pull/26789#issuecomment-6857015448) | 716.26 -> 710.34 ms (-0.83%) | 732.37 -> 725.23 ms (-0.97%) | Q7 minimum only | | [TPC-DS](https://github.com/apache/datafusion/pull/26789#issuecomment-6857053894) | 8774.66 -> 8685.77 ms (-1.01%) | 8921.44 -> 8861.27 ms (-0.67%) | Q42, Q81, and Q86 mean only | | [partitioned ClickBench](https://github.com/apache/datafusion/pull/26789#issuecomment-6857130415) | 18951.62 -> 19006.22 ms (+0.29%) | 19387.67 -> 19510.02 ms (+0.63%) | Q19/Q36 minimum only; Q18/Q26/Q35 mean only | TPC-DS Q47 and Q57 are the clear repeatable wins: both improve by about 9% in the minimum table and 10% in the mean table. No slower query reproduces above 5% in both the minimum and mean tables. Q19 even changes direction: its minimum is 7% slower while its mean is 43% faster. The raw output is not literally free of red labels. Q7, Q19, and Q36 are minimum-only threshold crossings; Q81/Q86 and the ClickBench mean-only flags have large changed-side tail outliers; Q42 is a borderline 5% mean-only crossing. So this run is effectively flat outside the intended TPC-DS wins, but it does not satisfy a strict "zero bot flags" gate. There are three commits on current `main` that are not in the PR: documentation, an opt-in CSV header feature (these suites use Parquet), and a behavior-preserving `EnforceDistribution` refactor. None changes dictionary aggregation, Parquet dictionary reads, or the benchmark configuration. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
