Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/24556 )
Change subject: IMPALA-15127: Support HBO for UnionNode cardinality ...................................................................... Patch Set 11: (4 comments) http://gerrit.cloudera.org:8080/#/c/24556/8/fe/src/main/java/org/apache/impala/planner/UnionNode.java File fe/src/main/java/org/apache/impala/planner/UnionNode.java: http://gerrit.cloudera.org:8080/#/c/24556/8/fe/src/main/java/org/apache/impala/planner/UnionNode.java@376 PS8, Line 376: if (limit_ > 0) sb.append("LIMIT:").append(limit_).append("|"); > I can't say for sure if it's better than nothing because if the cardinality Ack http://gerrit.cloudera.org:8080/#/c/24556/8/fe/src/main/java/org/apache/impala/planner/UnionNode.java@391 PS8, Line 391: public void appendScanInputStats(TPlanNodeRun execStats) { > This approach sounds reasonable if the fallback order is used for both the Done. Added an e2e test. http://gerrit.cloudera.org:8080/#/c/24556/8/fe/src/main/java/org/apache/impala/planner/UnionNode.java@415 PS8, Line 415: populateHboThriftFields(msg, serialCtx); > Changing the log level would only hide the failed write. Since PS10 fixes t Good points! These verify the if-branches that we skip HBO. Added the FE test and e2e test. http://gerrit.cloudera.org:8080/#/c/24556/8/fe/src/main/java/org/apache/impala/service/HistoricalStats.java File fe/src/main/java/org/apache/impala/service/HistoricalStats.java: http://gerrit.cloudera.org:8080/#/c/24556/8/fe/src/main/java/org/apache/impala/service/HistoricalStats.java@118 PS8, Line 118: for (int i = 0; i < runs.size(); i++) { > Agreed that keeping 120 is useful. I wasn't suggesting that it must be remo Searching newest-to-oldest still have problems. E.g. when the list has only 110 and then 122 (>110*1.1) comes, the list become [110, 122]. Then 122 is newer and it's able to serve 110 since it covers the range of [122 * 0,9, 122 * 1.1] = [109, 134]. Preferring the closest match is hard to define since each run has multiple scans and the scan matching could compare numRows, catalogVersion and totalFileSize. I think the current approach is simple but it works after enough runs given that - The similar one will be removed in the write path and the latest one will be added to the list. - The list has a fixed capacity so old/stale runs won't pile up. How about leaving this for future improvement when we find it worths the effort? -- To view, visit http://gerrit.cloudera.org:8080/24556 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie228f530bdcb171d3b717966673164bf9a4c45c8 Gerrit-Change-Number: 24556 Gerrit-PatchSet: 11 Gerrit-Owner: Quanlong Huang <[email protected]> Gerrit-Reviewer: Aleksandr Efimov <[email protected]> Gerrit-Reviewer: Aman Sinha <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Steve Carlin <[email protected]> Gerrit-Comment-Date: Wed, 12 Aug 2026 14:59:54 +0000 Gerrit-HasComments: Yes
