Peter Rozsa has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/20133 )
Change subject: IMPALA-12089: Be able to skip pushing down a subset of the predicates ...................................................................... IMPALA-12089: Be able to skip pushing down a subset of the predicates This change adds a predicate filtering mechanism at planning time that locates Impala's predicates in the residual expressions from Iceberg planning. By locating all residual expressions, the remainder expression set can be calculated. If there's no residual expression, the behaviour is the same as before, all predicate pushdown is skipped. If there is a residual expression, an Iceberg expression visitor implementation tries to locate each Impala predicate in the expression tree that the Iceberg planning returned as a residual expression. If the visitor's result is mismatched (the number of expressions does not match the number of located expressions), then it resorts to the original behaviour, all predicates are pushed down to the scanners. The visitor implementation handles IN, NULL and binary expressions, and does not handle compound expressions created by Impala, and aggregates, startsWith, isNan expressions by Iceberg. A new query option is added: iceberg_predicate_subsetting. The query option's default value is true. It can be turned off by setting it to false. Tests: - unit tests added for ImpalaExpressionVisitor - planner test cases added for disabled mode - core tests passed Change-Id: I597f69ad03ecaf9e304613ef934654e3d9614ae8 --- M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaService.thrift M common/thrift/Query.thrift A fe/src/main/java/org/apache/impala/analysis/ImpalaExpressionVisitor.java M fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java A fe/src/test/java/org/apache/impala/analysis/ImpalaExpressionVisitorTest.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java A testdata/workloads/functional-planner/queries/PlannerTest/iceberg-predicates-disabled-subsetting.test M testdata/workloads/functional-planner/queries/PlannerTest/iceberg-predicates.test M testdata/workloads/functional-planner/queries/PlannerTest/iceberg-v2-tables.test M testdata/workloads/functional-planner/queries/PlannerTest/tablesample.test 12 files changed, 656 insertions(+), 38 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/20133/3 -- To view, visit http://gerrit.cloudera.org:8080/20133 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I597f69ad03ecaf9e304613ef934654e3d9614ae8 Gerrit-Change-Number: 20133 Gerrit-PatchSet: 3 Gerrit-Owner: Peter Rozsa <pro...@cloudera.com> Gerrit-Reviewer: Gabor Kaszab <gaborkas...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Peter Rozsa <pro...@cloudera.com>
