alamb commented on issue #6278: URL: https://github.com/apache/arrow-datafusion/issues/6278#issuecomment-1540211578
I believe the slowdown began in https://github.com/apache/arrow-datafusion/pull/6103 191af8dcd7c4ee756b7d98e06bd7b40dc23202b7, as suspected by @mingmwang Specifically: * 87a67d4b190d5849969ead52fe75f4d6fccbbdec the query takes 1.85s for me * 191af8dcd7c4ee756b7d98e06bd7b40dc23202b7 (the next commit) the query takes 6.23s Here is the data: (I built datafusion-cli at different revisions) ``` + /home/alamb/2023-05-09-tpch-perf-drop/datafusion-cli-87a67d4b190d5849969ead52fe75f4d6fccbbdec -f q1.sql DataFusion CLI v23.0.0 +--------------+--------------+-------------+-----------------+-------------------+---------------------+-----------+--------------+----------+-------------+ | l_returnflag | l_linestatus | sum_qty | sum_base_price | sum_disc_price | sum_charge | avg_qty | avg_price | avg_disc | count_order | +--------------+--------------+-------------+-----------------+-------------------+---------------------+-----------+--------------+----------+-------------+ | A | F | 37734107.00 | 56586554400.73 | 53758257134.8700 | 55909065222.827692 | 25.522005 | 38273.129734 | 0.049985 | 1478493 | | N | F | 991417.00 | 1487504710.38 | 1413082168.0541 | 1469649223.194375 | 25.516471 | 38284.467760 | 0.050093 | 38854 | | N | O | 74476040.00 | 111701729697.74 | 106118230307.6056 | 110367043872.497010 | 25.502226 | 38249.117988 | 0.049996 | 2920374 | | R | F | 37719753.00 | 56568041380.90 | 53741292684.6040 | 55889619119.831932 | 25.505793 | 38250.854626 | 0.050009 | 1478870 | +--------------+--------------+-------------+-----------------+-------------------+---------------------+-----------+--------------+----------+-------------+ 4 rows in set. Query took 1.858 seconds. SHA: 191af8dcd7c4ee756b7d98e06bd7b40dc23202b7 + /home/alamb/2023-05-09-tpch-perf-drop/datafusion-cli-191af8dcd7c4ee756b7d98e06bd7b40dc23202b7 -f q1.sql DataFusion CLI v23.0.0 +--------------+--------------+-------------+-----------------+-------------------+---------------------+-----------+--------------+----------+-------------+ | l_returnflag | l_linestatus | sum_qty | sum_base_price | sum_disc_price | sum_charge | avg_qty | avg_price | avg_disc | count_order | +--------------+--------------+-------------+-----------------+-------------------+---------------------+-----------+--------------+----------+-------------+ | A | F | 37734107.00 | 56586554400.73 | 53758257134.8700 | 55909065222.827692 | 25.522005 | 38273.129734 | 0.049985 | 1478493 | | N | F | 991417.00 | 1487504710.38 | 1413082168.0541 | 1469649223.194375 | 25.516471 | 38284.467760 | 0.050093 | 38854 | | N | O | 74476040.00 | 111701729697.74 | 106118230307.6056 | 110367043872.497010 | 25.502226 | 38249.117988 | 0.049996 | 2920374 | | R | F | 37719753.00 | 56568041380.90 | 53741292684.6040 | 55889619119.831932 | 25.505793 | 38250.854626 | 0.050009 | 1478870 | +--------------+--------------+-------------+-----------------+-------------------+---------------------+-----------+--------------+----------+-------------+ 4 rows in set. Query took 6.233 seconds. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
