Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/19534 )
Change subject: IMPALA-11701 Part1: Don't push down predicates to scanner if already applied by Iceberg ...................................................................... Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/19534/1/fe/src/main/java/org/apache/impala/planner/IcebergScanNode.java File fe/src/main/java/org/apache/impala/planner/IcebergScanNode.java: http://gerrit.cloudera.org:8080/#/c/19534/1/fe/src/main/java/org/apache/impala/planner/IcebergScanNode.java@64 PS1, Line 64: private List<Expr> skippedConjuncts; > I dropped it then. Though if we have the following case: part_id_col = 42 AND non_part_col < 10 (let's assume table is IDENTITY-partitioned by part_id_col) Then Iceberg is able to filter out partitions via part_id_col = 42, but we cannot skip evaluating part_id_col = 42 in the scanners because there is still a residual expression (non_part_col < 10). And now, if 'part_id_col = 42' will still be used to calculate the cardinality estimates, and it will probably cause very bad estimations. So maybe we can only retire 'nonIdentityConjuncts_' when have a better logic to determine which predicate is part of the Iceberg residual and which isn't. Sorry for the inconvenience. -- To view, visit http://gerrit.cloudera.org:8080/19534 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icfa80ce469cecfcfbcd0dcb595a6b04b7027285b Gerrit-Change-Number: 19534 Gerrit-PatchSet: 1 Gerrit-Owner: Gabor Kaszab <gaborkas...@cloudera.com> Gerrit-Reviewer: Daniel Becker <daniel.bec...@cloudera.com> Gerrit-Reviewer: Gabor Kaszab <gaborkas...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Comment-Date: Fri, 31 Mar 2023 09:54:02 +0000 Gerrit-HasComments: Yes
