lipeng...@apache.org has posted comments on this change. ( http://gerrit.cloudera.org:8080/19416 )
Change subject: IMPALA-11798: Property 'external.table.purge' should not be ignored when CREATE Iceberg tables ...................................................................... Patch Set 2: (1 comment) Thanks for comments! http://gerrit.cloudera.org:8080/#/c/19416/1/fe/src/main/java/org/apache/impala/catalog/Table.java File fe/src/main/java/org/apache/impala/catalog/Table.java: http://gerrit.cloudera.org:8080/#/c/19416/1/fe/src/main/java/org/apache/impala/catalog/Table.java@191 PS1, Line 191: TRUE > Thank you for the explanation, I was not aware that the entire external.tab There are two scenarios(TL;DR): <-- Dividing line --> 1 execute 'create table' statement Query1: create table ice_tbl (col_1 int) stored as iceberg; Query2: show create table ice_tbl "CREATE EXTERNAL TABLE xyz.ice_tbl ( col_1 INT NULL ) STORED AS ICEBERG LOCATION 'hdfs://localhost:20500/test-warehouse/xyz.db/ice_tbl' TBLPROPERTIES ('external.table.purge'='TRUE', ...)" 'external.table.purge' will be true by default, because without 'external' keyword Hive will consider like we are trying for managed table, so Hive set it to true by default for CREATE TABLE statement without 'external' keyword. <-- Dividing line --> 2 execute 'create external table' statement Query1: create external table ice_tbl_ext stored as iceberg location '/data/ice_tbl_ext' tblproperties('iceberg.catalog' = 'hadoop.tables'); Query2: show create table ice_v2_update_all_rows "CREATE EXTERNAL TABLE xyz.ice_tbl_ext ( col_int INT NOT NULL, col_str STRING NOT NULL ) STORED AS ICEBERG LOCATION 'hdfs://localhost:20500/data/ice_tbl_ext' TBLPROPERTIES (...)" 'external.table.purge' will be empty, but if we delete this table 'ice_tbl_ext', the data under 'hdfs://localhost:20500/data/ice_tbl_ext' will not be deleted, because "hive.external.table.purge.default" is false https://github.com/apache/hive/blob/master/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java#L3660. <-- Dividing line --> So we set TBL_PROP_EXTERNAL_TABLE_PURGE_DEFAULT -> TRUE is consistent with Hive, because it only applies to CREATE TABLE statement without 'external' keyword. -- To view, visit http://gerrit.cloudera.org:8080/19416 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2649dd38fbe050044817d6c425ef447245aa2829 Gerrit-Change-Number: 19416 Gerrit-PatchSet: 2 Gerrit-Owner: Anonymous Coward <lipeng...@apache.org> Gerrit-Reviewer: Anonymous Coward <lipeng...@apache.org> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Tamas Mate <tma...@apache.org> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Comment-Date: Wed, 01 Feb 2023 11:38:34 +0000 Gerrit-HasComments: Yes
