Tamas Mate has posted comments on this change. ( http://gerrit.cloudera.org:8080/18531 )
Change subject: IMPALA-10453: Support file pruning via runtime filters on Iceberg ...................................................................... Patch Set 5: (2 comments) http://gerrit.cloudera.org:8080/#/c/18531/4/fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java File fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java: http://gerrit.cloudera.org:8080/#/c/18531/4/fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java@268 PS4, Line 268: partField.getTransformType() == TIcebergPartitionTransformType.IDENTITY) > this extra condition disables runtime filter generation for all partition t It is because FileMetadtaUtils::CreateTemplateTuple only resolves IDENTITY transform currently. Other transforms create empty tuples causing the runtime filter to always reject the files. Do you think it improving CreateTemplateTuple to support other partition transforms would be a feasible solution? By gap in testing, you mean that partition runtime filter tests should cover all transforms? http://gerrit.cloudera.org:8080/#/c/18531/4/testdata/workloads/functional-query/queries/QueryTest/iceberg-in-predicate-push-down.test File testdata/workloads/functional-query/queries/QueryTest/iceberg-in-predicate-push-down.test: http://gerrit.cloudera.org:8080/#/c/18531/4/testdata/workloads/functional-query/queries/QueryTest/iceberg-in-predicate-push-down.test@a59 PS4, Line 59: > Why do we remove these sections? This was an early fix that remained, disabling runtime filters should have fixed this as well. Removed it. -- To view, visit http://gerrit.cloudera.org:8080/18531 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7762e1238bdf236b85d2728881a402a2bb41f36a Gerrit-Change-Number: 18531 Gerrit-PatchSet: 5 Gerrit-Owner: Tamas Mate <tma...@apache.org> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Gergely Fürnstáhl <gfurnst...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Tamas Mate <tma...@apache.org> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Comment-Date: Thu, 16 Jun 2022 10:04:23 +0000 Gerrit-HasComments: Yes
