Hello Zoltan Borok-Nagy, Noemi Pap-Takacs, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23783
to look at the new patch set (#10).
Change subject: IMPALA-14321: Add BINARY partition transform to Iceberg tables
......................................................................
IMPALA-14321: Add BINARY partition transform to Iceberg tables
With this change we add support for IDENTITY, TRUNCATE and BUCKET
partition transformation functions with binary parameter to Iceberg
tables.
Flatbuffer schema has changed, because when reading a string,
flatbuffers tries to enforce UTF-8 encoding, which fails for arbitrary
binary data. FbIcebergDataFile's raw_partition_fields is an array of
ubyte arrays from now on.
Testing:
- Added TestBinary() in iceberg-functions-test.cc with truncate width
edge cases
- Extended iceberg-partitioned-insert-*.test files with binary_col
partition tests
- Verified partition pruning works correctly for BINARY predicates
(NumFileMetadataRead metrics)
Generated-by: Github Copilot (Claude Sonnet 4.5)
Change-Id: I5fd1ef382aa064dad55445dea00fbd39caeca1d3
---
M be/src/exec/file-metadata-utils.cc
M be/src/exprs/iceberg-functions-ir.cc
M be/src/exprs/iceberg-functions-test.cc
M be/src/exprs/iceberg-functions.h
M be/src/runtime/dml-exec-state.cc
M common/fbs/IcebergObjects.fbs
M common/function-registry/impala_functions.py
M fe/src/main/java/org/apache/impala/analysis/IcebergPartitionField.java
M fe/src/main/java/org/apache/impala/analysis/ShowFilesStmt.java
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/common/IcebergPredicateConverter.java
M fe/src/main/java/org/apache/impala/util/IcebergUtil.java
M
testdata/workloads/functional-query/queries/QueryTest/iceberg-drop-partition.test
M
testdata/workloads/functional-query/queries/QueryTest/iceberg-partitioned-insert-default.test
M
testdata/workloads/functional-query/queries/QueryTest/iceberg-partitioned-insert-v1.test
M
testdata/workloads/functional-query/queries/QueryTest/iceberg-partitioned-insert-v2.test
M
testdata/workloads/functional-query/queries/QueryTest/iceberg-show-files-partition.test
17 files changed, 426 insertions(+), 131 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/23783/10
--
To view, visit http://gerrit.cloudera.org:8080/23783
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5fd1ef382aa064dad55445dea00fbd39caeca1d3
Gerrit-Change-Number: 23783
Gerrit-PatchSet: 10
Gerrit-Owner: Daniel Vanko <[email protected]>
Gerrit-Reviewer: Daniel Vanko <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Noemi Pap-Takacs <[email protected]>
Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
