Csaba Ringhofer has uploaded this change for review. ( http://gerrit.cloudera.org:8080/22995
Change subject: POC IMPALA-14123: Allow forcing predicate push down to Iceberg ...................................................................... POC IMPALA-14123: Allow forcing predicate push down to Iceberg Since IMPALA-11591 Impala tries to avoid pushing down predicates to Iceberg unless it is necessary (timetravel) or is likely to be useful (at least 1 partition column is involved in predicates). While this makes planning faster, it may miss opportunities to skip files during planning. This patch adds table property impala.iceberg.push_down_hint that expects a comma separated list of column names and leads to push down to Iceberg when there is a predicate on any of these columns. Users can set this manually, while in the future Impala or other tools may be able to set it automatically, e.g. during COMPUTE STATS if there are many files with non-overlappin min/max stats for a given column. Note that in most cases when Iceberg can skip files the Parquet/ORC scanner would also skip most of the data based on stat filtering. The benefit of doing it during planning is reading less headers and a "smaller" query plan. Change-Id: I8eb4ab5204c20b3991fdf305d7317f4023904a0f --- M fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java A testdata/workloads/functional-query/queries/QueryTest/iceberg-predicate-push-down-hint.test M tests/query_test/test_iceberg.py 3 files changed, 95 insertions(+), 16 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/95/22995/1 -- To view, visit http://gerrit.cloudera.org:8080/22995 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I8eb4ab5204c20b3991fdf305d7317f4023904a0f Gerrit-Change-Number: 22995 Gerrit-PatchSet: 1 Gerrit-Owner: Csaba Ringhofer <[email protected]>
