Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/20887 )
Change subject: IMPALA-12636: Reload filemetadata for AlterTable event of type truncate ...................................................................... Patch Set 1: (6 comments) http://gerrit.cloudera.org:8080/#/c/20887/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/20887/1//COMMIT_MSG@15 PS1, Line 15: Note: Alter table event for an external table generated by the truncate : operation from Impala cannot be identified if it's a truncate op or not Is it only true on tables that are not being replicated? For tables that are being replicated, Impala does invoke the HMS truncateTable API so events will be generated: https://github.com/apache/impala/blob/a2b8aed2c2d4c1bb25ed9626a2b014b79ec741ad/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java#L3314-L3318 Or are those events don't set the isTruncateOp field? http://gerrit.cloudera.org:8080/#/c/20887/1//COMMIT_MSG@18 PS1, Line 18: generated one impala cluster is consumed bu other impala clusters. nit: "generated" -> "generated from" "bu" -> "by" http://gerrit.cloudera.org:8080/#/c/20887/1/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java: http://gerrit.cloudera.org:8080/#/c/20887/1/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@1565 PS1, Line 1565: if (canBeSkipped() && !isTruncateOp_) { nit: check 'isTruncateOp_' first since it's more light-weight. http://gerrit.cloudera.org:8080/#/c/20887/1/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@1570 PS1, Line 1570: skipFileMetadataReload_ = canSkipFileMetadataReload(tableBefore_, tableAfter_) : && !isTruncateOp_; nit: check 'isTruncateOp_' first since it's more light-weight. http://gerrit.cloudera.org:8080/#/c/20887/1/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@2207 PS1, Line 2207: AlterPartitionMessage alterPartitionMessage = What about partition level truncate operations? AlterPartitionMessage also has the isTruncateOp field. http://gerrit.cloudera.org:8080/#/c/20887/1/tests/custom_cluster/test_events_custom_configs.py File tests/custom_cluster/test_events_custom_configs.py: http://gerrit.cloudera.org:8080/#/c/20887/1/tests/custom_cluster/test_events_custom_configs.py@1096 PS1, Line 1096: create table {0}.{1} (i int)""".format(unique_database, tbl_name)) Can we also test on partitioned tables and transactional tables? Also test on partition-level truncate: TRUNCATE [TABLE] table_name [PARTITION partition_spec]; -- To view, visit http://gerrit.cloudera.org:8080/20887 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I53bb80c294623eec7c79d9f30f410771386c6b75 Gerrit-Change-Number: 20887 Gerrit-PatchSet: 1 Gerrit-Owner: Sai Hemanth Gantasala <saihema...@cloudera.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Sai Hemanth Gantasala <saihema...@cloudera.com> Gerrit-Comment-Date: Thu, 11 Jan 2024 23:55:19 +0000 Gerrit-HasComments: Yes
