Peter Rozsa has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/20515 )
Change subject: IMPALA-12243: Add support for DROP PARTITION for Iceberg tables ...................................................................... IMPALA-12243: Add support for DROP PARTITION for Iceberg tables This change adds support for DROP PARTITION operation for Iceberg tables. Users can now execute 'ALTER TABLE table DROP PARTITION (partition_filtering_expression)' statements where 'partition_filtering_expression' consists of at least one binary predicate, IN predicate, IS NULL predicate or arbitrary logical combination of these, where the predicate's term is a partitioned column decorated with a partition transform, for example: 'day(column)' or simply column. To select partitions with mutating transforms (non-identity transforms), the user must provide the transform in the selection, for example: if column 'a' is partitioned as year(a), the user must filter it with explicitly providing the transform, like (year(a) = "2012"). This is a requirement for Iceberg's planFiles API. If the column is filtered without the transform, the filtering of datafiles with strict projection results in ambiguous results. The binary predicate's operand can be basic comparison operators: =, !=, <, >, <=, >=. The IN and IS NULL predicates can be inverted with the NOT keyword. Only constant literals are accepted currently for these predicates; for example: (identity_string = "string") is accepted, (identity_string = concat(identity_string, "another_string") is not accepted. Tests: - unit tests for IcebergUtil.getDateTransformValue - analyzer tests for negative cases, simple cases - e2e tests for every transform type, schema evolution and rollback Change-Id: I2a768ba2966f570454687e02e4e6d67df46741f9 --- M common/thrift/CatalogObjects.thrift M common/thrift/JniCatalog.thrift M fe/src/main/java/org/apache/impala/analysis/AlterTableDropPartitionStmt.java A fe/src/main/java/org/apache/impala/analysis/IcebergPartitionExpr.java M fe/src/main/java/org/apache/impala/analysis/IcebergPartitionField.java M fe/src/main/java/org/apache/impala/analysis/PartitionSet.java M fe/src/main/java/org/apache/impala/analysis/PartitionSpecBase.java M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergCtasTarget.java M fe/src/main/java/org/apache/impala/catalog/local/LocalIcebergTable.java A fe/src/main/java/org/apache/impala/common/IcebergPartitionPredicateConverter.java A fe/src/main/java/org/apache/impala/common/IcebergPredicateConverter.java M fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/DescribeResultFactory.java M fe/src/main/java/org/apache/impala/service/IcebergCatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/IcebergUtil.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java M fe/src/test/java/org/apache/impala/util/IcebergUtilTest.java A testdata/workloads/functional-query/queries/QueryTest/iceberg-drop-partition.test M testdata/workloads/functional-query/queries/QueryTest/iceberg-negative.test M tests/query_test/test_iceberg.py 23 files changed, 1,496 insertions(+), 330 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/15/20515/6 -- To view, visit http://gerrit.cloudera.org:8080/20515 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I2a768ba2966f570454687e02e4e6d67df46741f9 Gerrit-Change-Number: 20515 Gerrit-PatchSet: 6 Gerrit-Owner: Peter Rozsa <pro...@cloudera.com> Gerrit-Reviewer: Gabor Kaszab <gaborkas...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Peter Rozsa <pro...@cloudera.com> Gerrit-Reviewer: Tamas Mate <tma...@apache.org> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com>
