Hello Zoltan Borok-Nagy, Noemi Pap-Takacs, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/22995
to look at the new patch set (#3).
Change subject: IMPALA-14123: Allow forcing predicate push down to Iceberg
......................................................................
IMPALA-14123: Allow forcing predicate push down to Iceberg
Since IMPALA-11591 Impala tries to avoid pushing down predicates to
Iceberg unless it is necessary (timetravel) or is likely to be useful
(at least 1 partition column is involved in predicates). While this
makes planning faster, it may miss opportunities to skip files during
planning.
This patch adds table property impala.iceberg.push_down_hint that
expects a comma separated list of column names and leads to push
down to Iceberg when there is a predicate on any of these columns.
Users can set this manually, while in the future Impala or other tools
may be able to set it automatically, e.g. during COMPUTE STATS if
there are many files with non-overlapping min/max stats for a given
column.
Note that in most cases when Iceberg can skip files the Parquet/ORC
scanner would also skip most of the data based on stat filtering. The
benefit of doing it during planning is reading less footers and a
"smaller" query plan.
Change-Id: I8eb4ab5204c20b3991fdf305d7317f4023904a0f
---
M fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java
A
testdata/workloads/functional-query/queries/QueryTest/iceberg-predicate-push-down-hint.test
M tests/query_test/test_iceberg.py
3 files changed, 100 insertions(+), 17 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/95/22995/3
--
To view, visit http://gerrit.cloudera.org:8080/22995
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I8eb4ab5204c20b3991fdf305d7317f4023904a0f
Gerrit-Change-Number: 22995
Gerrit-PatchSet: 3
Gerrit-Owner: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Noemi Pap-Takacs <[email protected]>
Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
