alamb opened a new pull request #8553: URL: https://github.com/apache/arrow/pull/8553
This PR is based on https://github.com/apache/arrow/pull/8503 from jorgecarlieto Note that this does not add any new dependencies (pin_project_lite is already required indirectly via tokio) Aggregate Pperformance, as measured with the following command, reports improvement between 5 and 15 percent on my laptop, though the variance is quite high. ``` git checkout master && \ cargo bench --bench aggregate_query_sql && \ git checkout alamb/ARROW-10366-collect-in-parallel && \ cargo bench --bench aggregate_query_sql ``` # A collapsible section with markdown <details> <summary>Click for performance details</summary> aggregate_query_no_group_by 15 12 time: [715.38 us 717.64 us 720.07 us] change: [-5.6870% -4.7352% -3.6834%] (p = 0.00 < 0.05) Performance has improved. Found 8 outliers among 100 measurements (8.00%) 2 (2.00%) high mild 6 (6.00%) high severe Benchmarking aggregate_query_group_by 15 12: Warming up for 3.0000 s Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 13.7s or reduce sample count to 40. aggregate_query_group_by 15 12 time: [2.4874 ms 2.5029 ms 2.5173 ms] change: [-13.513% -12.381% -11.184%] (p = 0.00 < 0.05) Performance has improved. Found 5 outliers among 100 measurements (5.00%) 3 (3.00%) high mild 2 (2.00%) high severe Benchmarking aggregate_query_group_by_with_filter 15 12: Warming up for 3.0000 s Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 10.2s or reduce sample count to 40. aggregate_query_group_by_with_filter 15 12 time: [1.9724 ms 1.9856 ms 1.9982 ms] change: [-14.759% -10.342% -6.5522%] (p = 0.00 < 0.05) Performance has improved. Found 5 outliers among 100 measurements (5.00%) 2 (2.00%) high mild 3 (3.00%) high severe </details> ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
