Hello Andrew Sherman, Tamas Mate, Gabor Kaszab, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19601
to look at the new patch set (#3).
Change subject: IMPALA-11964: Make sure Impala returns error for Iceberg tables
with equality deletes
......................................................................
IMPALA-11964: Make sure Impala returns error for Iceberg tables with equality
deletes
Impala only supports position deletes currently. It should raise an
error when equality deletes are encountered.
We already had a check for this when the query was planned by Iceberg.
But when we were using cached metadata the check was missing. This means
that Impala could return bogus results in the presence of equality
delete files. This patch adds check for the latter case as well.
Tables with equality delete files are still loadable by Impala, and
users can still query snapshots of it if they don't have equality
deletes.
Testing:
* added e2e tests
Change-Id: I14d7116692c0e47d0799be650dc323811e2ee0fb
---
M common/thrift/CatalogObjects.thrift
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergContentFileStore.java
M fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java
M testdata/data/README
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_delete_equality/data/00000-0-fb178c51-b12a-4c5f-a66e-a8e9375daeba-00001.parquet
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_delete_equality/data/00191-4-6e780302-527b-4911-8c6e-88d416adac57-00001.parquet
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_delete_equality/metadata/0eadf173-0c84-4378-a9d0-5d7f47183978-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_delete_equality/metadata/8cbef400-daea-478a-858a-2baf2438f644-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_delete_equality/metadata/snap-5725822353600261755-1-0eadf173-0c84-4378-a9d0-5d7f47183978.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_delete_equality/metadata/snap-6816997371555012807-1-8cbef400-daea-478a-858a-2baf2438f644.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_delete_equality/metadata/v1.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_delete_equality/metadata/v2.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_delete_equality/metadata/version-hint.text
M testdata/datasets/functional/functional_schema_template.sql
M testdata/datasets/functional/schema_constraints.csv
M testdata/workloads/functional-query/queries/QueryTest/iceberg-negative.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-query.test
18 files changed, 269 insertions(+), 19 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/19601/3
--
To view, visit http://gerrit.cloudera.org:8080/19601
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I14d7116692c0e47d0799be650dc323811e2ee0fb
Gerrit-Change-Number: 19601
Gerrit-PatchSet: 3
Gerrit-Owner: Zoltan Borok-Nagy <borokna...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <asher...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <gaborkas...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tma...@apache.org>
