Arnab Karmakar has uploaded this change for review. ( http://gerrit.cloudera.org:8080/24973
Change subject: IMPALA-15365: Iceberg UUID partition transform support ...................................................................... IMPALA-15365: Iceberg UUID partition transform support Allow only the identity, bucket(N, col) and void partition transforms on UUID columns, and reject the others at analysis time for CREATE TABLE and ALTER TABLE ... SET PARTITION SPEC. Iceberg compares UUIDs as signed longs, while Parquet min/max bounds use unsigned byte order, so Iceberg can skip data files that contain matching rows. UUID value predicates are therefore not pushed to Iceberg and are evaluated by the scan. IS NULL and IS NOT NULL are still pushed. DROP PARTITION, SHOW FILES ... PARTITION and SHOW PARTITIONS ... WHERE accept = and IN on UUID columns that are identity partitioned in every partition spec. Iceberg matches these values exactly against the partition values, so byte order does not matter. Other UUID comparisons are rejected in these statements. Testing: - FE tests: identity, bucket and void are accepted on UUID columns and the other transforms are rejected, for CREATE TABLE and SET PARTITION SPEC. - e2e tests on the new iceberg_uuid_test_part fixture (identity and bucket(4) partitions, including a data file whose values are on both sides of the sign bit) for reads, scan predicates, EXPLAIN, DROP PARTITION, SHOW FILES and SHOW PARTITIONS. - Trino interop tests for partition evolution from identity to bucket and for DROP PARTITION on Trino-written identity partitions. - Added the iceberg_uuid_test_part fixture (Iceberg V3, identity(uuid_identity), bucket(4, uuid_bucket)), generated with the Iceberg Java API (version 1.11.0). The test creates it from testdata/data instead of functional dataload. Change-Id: I1dc3af4418285d6ea1d489912419be470e668f29 Assisted-by: Claude Opus 5.5 (Cursor) --- M fe/src/main/java/org/apache/impala/analysis/AlterTableDropPartitionStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterTableSetPartitionSpecStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java M fe/src/main/java/org/apache/impala/analysis/IcebergPartitionField.java M fe/src/main/java/org/apache/impala/analysis/ShowFilesStmt.java M fe/src/main/java/org/apache/impala/analysis/ShowStatsStmt.java M fe/src/main/java/org/apache/impala/common/IcebergPartitionPredicateConverter.java M fe/src/main/java/org/apache/impala/common/IcebergPredicateConverter.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M testdata/data/README A testdata/data/iceberg_test/iceberg_uuid/iceberg_uuid_test_part/data/uuid_identity=00000000-0000-0000-0000-000000000000/uuid_bucket_bucket=0/00000-0-d726ce5a-ae1c-4e5c-a605-f618c6c54fef-1-00001.parquet A testdata/data/iceberg_test/iceberg_uuid/iceberg_uuid_test_part/data/uuid_identity=12345678-1234-5678-1234-567812345678/uuid_bucket_bucket=1/00000-0-ad1a4a3c-c8cd-4286-91fe-745117986415-1-00001.parquet A testdata/data/iceberg_test/iceberg_uuid/iceberg_uuid_test_part/data/uuid_identity=ffffffff-ffff-ffff-ffff-ffffffffffff/uuid_bucket_bucket=2/00000-0-e4353b24-49a8-481a-93f5-d9da6e3a73db-1-00001.parquet A testdata/data/iceberg_test/iceberg_uuid/iceberg_uuid_test_part/data/uuid_identity=null/uuid_bucket_bucket=null/00000-0-e01cf2f3-87d8-48e4-b58b-f955c4f8d067-1-00001.parquet A testdata/data/iceberg_test/iceberg_uuid/iceberg_uuid_test_part/metadata/294c9816-66b5-462b-ad13-3fde3940f76a-m0.avro A testdata/data/iceberg_test/iceberg_uuid/iceberg_uuid_test_part/metadata/ca55c847-5112-4bc8-af05-27a42072f3dc-m0.avro A testdata/data/iceberg_test/iceberg_uuid/iceberg_uuid_test_part/metadata/cda91123-feaf-48f9-9d0e-e020e801e805-m0.avro A testdata/data/iceberg_test/iceberg_uuid/iceberg_uuid_test_part/metadata/eaf89305-befa-43b9-9431-de37f7ca995b-m0.avro A testdata/data/iceberg_test/iceberg_uuid/iceberg_uuid_test_part/metadata/snap-1870879193109290683-1-eaf89305-befa-43b9-9431-de37f7ca995b.avro A testdata/data/iceberg_test/iceberg_uuid/iceberg_uuid_test_part/metadata/snap-2499074418364805247-1-ca55c847-5112-4bc8-af05-27a42072f3dc.avro A testdata/data/iceberg_test/iceberg_uuid/iceberg_uuid_test_part/metadata/snap-456531118915677437-1-cda91123-feaf-48f9-9d0e-e020e801e805.avro A testdata/data/iceberg_test/iceberg_uuid/iceberg_uuid_test_part/metadata/snap-5662060091786121272-1-294c9816-66b5-462b-ad13-3fde3940f76a.avro A testdata/data/iceberg_test/iceberg_uuid/iceberg_uuid_test_part/metadata/v1.metadata.json A testdata/data/iceberg_test/iceberg_uuid/iceberg_uuid_test_part/metadata/v2.metadata.json A testdata/data/iceberg_test/iceberg_uuid/iceberg_uuid_test_part/metadata/v3.metadata.json A testdata/data/iceberg_test/iceberg_uuid/iceberg_uuid_test_part/metadata/v4.metadata.json A testdata/data/iceberg_test/iceberg_uuid/iceberg_uuid_test_part/metadata/v5.metadata.json A testdata/data/iceberg_test/iceberg_uuid/iceberg_uuid_test_part/metadata/version-hint.text M testdata/workloads/functional-query/queries/QueryTest/iceberg-trino-interop-uuid.test M testdata/workloads/functional-query/queries/QueryTest/iceberg-uuid-type.test M tests/query_test/test_iceberg.py 31 files changed, 492 insertions(+), 18 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/73/24973/1 -- To view, visit http://gerrit.cloudera.org:8080/24973 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I1dc3af4418285d6ea1d489912419be470e668f29 Gerrit-Change-Number: 24973 Gerrit-PatchSet: 1 Gerrit-Owner: Arnab Karmakar <[email protected]>
