hhhizzz commented on PR #24705: URL: https://github.com/apache/datafusion/pull/24705#issuecomment-5447696353
I ran a focused five-arm experiment using the same DataFusion #24705 branch and a common Arrow base: - Pushdown OFF - Base ON - 10852 capped Auto only - 10859 fusion only - Combined The Combined arm explicitly routes fused predicate results through capped Auto. I ran Q24–Q26 for five position-balanced rounds, 10 iterations each, discarding iteration 0 and using the median of the remaining iterations. | Query | OFF | Base ON | Capped | Fusion | Combined | |---|---:|---:|---:|---:|---:| | Q24 | 134.0 ms | 95.0 ms | 93.7 ms | 95.8 ms | 94.6 ms | | Q25 | 205.5 ms | 278.3 ms | 262.3 ms | 220.2 ms | 223.5 ms | | Q26 | 131.1 ms | 216.9 ms | 214.9 ms | 216.3 ms | 215.4 ms | The results suggest: - Q24’s improvement mainly comes from the DataFusion gating changes; neither Arrow optimization adds much. - Q25 benefits from both changes independently: capped Auto improves about 6%, while fusion improves about 21%. - Combined is effectively the same as Fusion-only, so I did not observe an additive Q25 benefit. - Q26 is unchanged, consistent with its two-column TopK key not matching the same-single-column fusion condition. - Q23’s major pushdown benefit was preserved. My interpretation is that the changes are architecturally complementary, but their Q25 gains overlap in practice. Fusion removes the dominant repeated decode, leaving too little selection-lowering cost for capped Auto to provide a separately measurable gain. All result captures were equivalent across the five arms, and the focused equivalence tests and Parquet clippy passed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
