alamb commented on PR #10859: URL: https://github.com/apache/arrow-rs/pull/10859#issuecomment-5892107366
I ran this and confirmed that for Q25 the performance of pushdown vs not pushdown is now the same (maybe even slightly faster) 🎉 ClickBench q25 on `hits_partitioned`, local run, 10 iterations per phase (ms): | Binary | Pushdown | Min | Max | Avg | |---|---|---:|---:|---:| | merge-base (991fd23) | off | 76 | 85 | 78.5 | | merge-base (991fd23) | on | 100 | 115 | 106.4 | | PR 10859 | off | 77 | 86 | 80.3 | | PR 10859 | on | 76 | 82 | 78.3 | <details> Test script: [q25.sql](https://github.com/user-attachments/files/32807242/q25.sql) Run with this PR (via https://github.com/apache/datafusion/pull/25832) ```shell andrewlamb@Andrews-MacBook-Pro-3:~/Downloads/10859$ ../datafusion-cli-test-arrow-pr-10859 -f q25.sql | grep Elapsed Elapsed 0.001 seconds. Elapsed 0.035 seconds. Elapsed 0.001 seconds. | ===== Elapsed lines below: PUSHDOWN OFF (q25) ===== | Elapsed 0.000 seconds. Elapsed 0.000 seconds. Elapsed 0.000 seconds. Elapsed 0.083 seconds. Elapsed 0.079 seconds. Elapsed 0.077 seconds. Elapsed 0.085 seconds. Elapsed 0.086 seconds. Elapsed 0.080 seconds. Elapsed 0.078 seconds. Elapsed 0.078 seconds. Elapsed 0.079 seconds. Elapsed 0.078 seconds. | ===== Elapsed lines below: PUSHDOWN ON (q25) ===== | Elapsed 0.000 seconds. Elapsed 0.000 seconds. Elapsed 0.000 seconds. Elapsed 0.077 seconds. Elapsed 0.076 seconds. Elapsed 0.077 seconds. Elapsed 0.078 seconds. Elapsed 0.080 seconds. Elapsed 0.077 seconds. Elapsed 0.078 seconds. Elapsed 0.078 seconds. Elapsed 0.080 seconds. Elapsed 0.082 seconds. ``` Run with merge-base of the PR ```shell andrewlamb@Andrews-MacBook-Pro-3:~/Downloads/10859$ ../datafusion-cli.991fd23dd0be0046af5945b8c3905612859b657a -f q25.sql | grep Elapsed Elapsed 0.001 seconds. Elapsed 0.035 seconds. Elapsed 0.001 seconds. | ===== Elapsed lines below: PUSHDOWN OFF (q25) ===== | Elapsed 0.000 seconds. Elapsed 0.000 seconds. Elapsed 0.000 seconds. Elapsed 0.082 seconds. Elapsed 0.085 seconds. Elapsed 0.079 seconds. Elapsed 0.077 seconds. Elapsed 0.077 seconds. Elapsed 0.077 seconds. Elapsed 0.077 seconds. Elapsed 0.077 seconds. Elapsed 0.078 seconds. Elapsed 0.076 seconds. | ===== Elapsed lines below: PUSHDOWN ON (q25) ===== | Elapsed 0.000 seconds. Elapsed 0.000 seconds. Elapsed 0.000 seconds. Elapsed 0.100 seconds. Elapsed 0.100 seconds. Elapsed 0.102 seconds. Elapsed 0.103 seconds. Elapsed 0.110 seconds. Elapsed 0.108 seconds. Elapsed 0.115 seconds. Elapsed 0.108 seconds. Elapsed 0.112 seconds. Elapsed 0.106 seconds. ``` </details> Also I looked at q35,q36 and q39 that the bot may have shown slightly slower, but could not reproduce it locally -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
