Tamas Mate has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/18688 )
Change subject: IMPALA-11362: Add expire snapshots functionality for Iceberg tables ...................................................................... IMPALA-11362: Add expire snapshots functionality for Iceberg tables Iceberg table modifications can create new table snapshots, these snapshots can be used to access an earlier version of the table. During the lifetime of a table the number of snapshots can accumulate and older versions can become obsolete as well. Iceberg API provides a safe solution to remove the snapshots that are not needed anymore, the operation is called ExpireSnapshots. This commit adds framework to execute Iceberg maintenance operation on tables and implements the call of an expire snapshots operation. The following statement becomes available for Iceberg tables: - ALTER TABLE <tbl> EXECUTE expire_snapshots(<timestamp>) ExpireSnapshots Iceberg API calls were meant to be chained, the calls are expireSnapshotId, expireOlderThan and retainLast. SQL is less suitable for chained calls, therefore this commit implements only the expireOlderThan functionality. However, in this case the retainLast call will fall back to its default value (1), this default can be configured with TableProperties.MIN_SNAPSHOTS_TO_KEEP. This commit also refactors the Iceberg e2e tests and introduces ImpalaTestSuite for the common methods that are used in test which compare snapshot versions in some way. Testing: - Added analysis unit tests. - Added e2e tests. Change-Id: Ideffee4964c18c85ca745bfb4eca08ec362416f3 --- M common/thrift/JniCatalog.thrift M fe/src/main/cup/sql-parser.cup A fe/src/main/java/org/apache/impala/analysis/AlterTableExecuteStmt.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/jflex/sql-scanner.flex M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java A tests/common/iceberg_test_suite.py M tests/query_test/test_iceberg.py 8 files changed, 339 insertions(+), 78 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/18688/5 -- To view, visit http://gerrit.cloudera.org:8080/18688 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ideffee4964c18c85ca745bfb4eca08ec362416f3 Gerrit-Change-Number: 18688 Gerrit-PatchSet: 5 Gerrit-Owner: Tamas Mate <tma...@apache.org> Gerrit-Reviewer: Gabor Kaszab <gaborkas...@cloudera.com> Gerrit-Reviewer: Gergely Fürnstáhl <gfurnst...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Tamas Mate <tma...@apache.org> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com>
