Hello Tamas Mate, lipeng...@apache.org, Gergely Fürnstáhl, Impala Public
Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19419
to look at the new patch set (#2).
Change subject: IMPALA-11826: Avoid calling planFiles() on Iceberg V2 tables
when there are no predicates
......................................................................
IMPALA-11826: Avoid calling planFiles() on Iceberg V2 tables when there are no
predicates
Similar to IMPALA-11591 but this Jira extends it to V2 tables. With
this patch we group data files into two categories in
IcebergContentFileStore:
* data files without deletes
* data files with deletes
With this information we can avoid calling planFiles() when planning
the scans of Iceberg tables. We can just set the lists of the file
descriptors based on IcebergContentFileStore then invoke the regular
planning methods.
iceberg-v2-tables.test had to be updated a bit because now we are
calculating the lengths of the file paths based on Impala's file
descriptor objects + table location, and not based on data file
information in Iceberg metadata (which has the file system prefix
stripped)
Testing:
* executed existing tests
* Updated plan tests
Change-Id: Ia46bd2dce248a9e096fc1c0bd914fc3fa4686fb0
---
M common/thrift/CatalogObjects.thrift
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergContentFileStore.java
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
A fe/src/main/java/org/apache/impala/catalog/iceberg/GroupedContentFiles.java
M fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java
M fe/src/main/java/org/apache/impala/util/IcebergUtil.java
M fe/src/test/java/org/apache/impala/catalog/local/LocalCatalogTest.java
M
testdata/workloads/functional-planner/queries/PlannerTest/iceberg-v2-tables.test
9 files changed, 279 insertions(+), 158 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/19/19419/2
--
To view, visit http://gerrit.cloudera.org:8080/19419
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia46bd2dce248a9e096fc1c0bd914fc3fa4686fb0
Gerrit-Change-Number: 19419
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <borokna...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <lipeng...@apache.org>
Gerrit-Reviewer: Gergely Fürnstáhl <gfurnst...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tma...@apache.org>
