andygrove commented on PR #26447: URL: https://github.com/apache/datafusion/pull/26447#issuecomment-6524763996
Thanks @adriangb for running the benchmarks. TPC-DS q72, the query this PR targets, is 1.87x faster: 1,821 ms to 973 ms. TPC-H and ClickBench don't change. Adding up the fastest run of each query: | Suite | main | this PR | Change | | --- | --- | --- | --- | | [TPC-DS SF1](https://github.com/apache/datafusion/pull/26447#issuecomment-6251343888) | 9,557 ms | 8,720 ms | -8.8% | | [TPC-H SF1](https://github.com/apache/datafusion/pull/26447#issuecomment-6251330865) | 779 ms | 780 ms | +0.2% | | [ClickBench, partitioned](https://github.com/apache/datafusion/pull/26447#issuecomment-6251368710) | 19,280 ms | 19,199 ms | -0.4% | q72 accounts for all of the TPC-DS change: the other 98 queries add up to 11 ms more than on main. Apart from q72 and q15, each query the bot marks as faster or slower is marked on only one of its two measures, the fastest run or the mean. These include TPC-H q6 and the ClickBench queries marked up to 1.61x slower on the mean. None of them has a join, so this PR doesn't change how they run. Two results are worth a closer look: - TPC-DS q15 is about 6% slower on both measures: 59.1 ms to 62.9 ms in the fastest run, and 61.3 ms to 64.9 ms on average. Its fastest run on this PR is about as slow as the slowest on main, 63.2 ms. - The peak memory of the TPC-DS run went from 1.9 GiB to 2.2 GiB, while TPC-H stayed at 1.1 GiB and ClickBench went from 18.3 GiB to 18.5 GiB. The runner measures it over the whole run, so it doesn't show which query it comes from. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
