Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/22094 )
Change subject: IMPALA-13533: Calcite CTE backend ...................................................................... Patch Set 61: (1 comment) http://gerrit.cloudera.org:8080/#/c/22094/61/be/src/scheduling/scheduler.cc File be/src/scheduling/scheduler.cc: http://gerrit.cloudera.org:8080/#/c/22094/61/be/src/scheduling/scheduler.cc@372 PS61, Line 372: // Extra destination instances are MT_DOP copies and must not replay the same CTE : // stream. > When do we have extra destination instances? Is that when we have too few f Yeah, that's generally the case. In Q76 it's the difference between 06:UNION 3 6 488.797us 552.514us 64.79K 124.33K 24.00 KB 0 |--16:HASH JOIN 3 6 133.909us 195.879us 170 173 37.12 KB 0 INNER JOIN, BROADCAST | |--F18:JOIN BUILD 3 3 833.473us 869.642us 3.90 MB 3.88 MB | | 28:EXCHANGE 3 3 17.525us 18.837us 85 173 48.00 KB 16.00 KB BROADCAST | | F05:EXCHANGE SENDER 1 1 152.806us 152.806us 5.69 KB 160.00 KB | | 15:HASH JOIN 1 1 197.086us 197.086us 85 173 25.12 KB 0 INNER JOIN, BROADCAST | | |--F19:JOIN BUILD 1 1 831.461us 831.461us 3.89 MB 3.88 MB | | | 27:EXCHANGE 1 1 24.023us 24.023us 85 173 24.00 KB 16.00 KB BROADCAST | | | F06:EXCHANGE SENDER 2 2 57.311us 67.822us 2.81 KB 96.00 KB | | | 14:SCAN HDFS 2 2 36.921ms 47.590ms 85 173 8.69 MB 32.00 MB tpcds_parquet.web_sales | | 13:CTE CONSUMER 1 1 3.703ms 3.703ms 83 73.05K 4.00 MB 4.00 MB cte_suggestion_0 | 12:CTE CONSUMER 3 6 358.892us 1.086ms 168 18.00K 4.00 MB 4.00 MB cte_suggestion_1 | F15:LOCAL MULTI SINK 1 1 0.000ns 0.000ns 0 1.00 KB | 03:CTE PRODUCER 1 1 72.220us 72.220us 0 18.00K 6.53 MB 6.00 MB cte_suggestion_1, CANCELLED | 04:SCAN HDFS 1 1 103.132ms 103.132ms 18.00K 18.00K 549.25 KB 16.00 MB tpcds_parquet.item and 06:UNION 3 6 519.304us 667.023us 64.71K 124.33K 24.00 KB 0 |--16:HASH JOIN 3 6 120.119us 158.345us 85 173 37.12 KB 0 INNER JOIN, BROADCAST | |--F18:JOIN BUILD 3 3 854.865us 961.155us 3.90 MB 3.88 MB | | 28:EXCHANGE 3 3 28.349us 34.914us 85 173 32.00 KB 16.00 KB BROADCAST | | F05:EXCHANGE SENDER 1 1 193.975us 193.975us 5.69 KB 160.00 KB | | 15:HASH JOIN 1 1 327.374us 327.374us 85 173 25.12 KB 0 INNER JOIN, BROADCAST | | |--F19:JOIN BUILD 1 1 791.643us 791.643us 3.89 MB 3.88 MB | | | 27:EXCHANGE 1 1 21.311us 21.311us 85 173 24.00 KB 16.00 KB BROADCAST | | | F06:EXCHANGE SENDER 2 2 50.865us 52.599us 2.81 KB 96.00 KB | | | 14:SCAN HDFS 2 2 63.245ms 64.655ms 85 173 8.69 MB 32.00 MB tpcds_parquet.web_sales | | 13:CTE CONSUMER 1 1 4.446ms 4.446ms 83 73.05K 4.00 MB 4.00 MB cte_suggestion_0 | 12:CTE CONSUMER 3 6 212.244us 1.004ms 84 18.00K 4.00 MB 4.00 MB cte_suggestion_1 | F15:LOCAL MULTI SINK 1 1 0.000ns 0.000ns 0 1.00 KB | 03:CTE PRODUCER 1 1 92.442us 92.442us 0 18.00K 6.53 MB 6.00 MB cte_suggestion_1, CANCELLED | 04:SCAN HDFS 1 1 87.426ms 87.426ms 18.00K 18.00K 549.25 KB 16.00 MB tpcds_parquet.item I forget why I tried round-robin before, it doesn't seem to be an issue now. I think a lot of queries masked the issue by deduplicating the extra rows at some point above the CTE. -- To view, visit http://gerrit.cloudera.org:8080/22094 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I48f16d495d4b37be97e6a913f0eb5b94d70e199a Gerrit-Change-Number: 22094 Gerrit-PatchSet: 61 Gerrit-Owner: Michael Smith <[email protected]> Gerrit-Reviewer: Anonymous Coward (816) Gerrit-Reviewer: Balazs Hevele <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Steve Carlin <[email protected]> Gerrit-Comment-Date: Thu, 13 Aug 2026 22:25:40 +0000 Gerrit-HasComments: Yes
