Vihang Karajgaonkar has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14750
Change subject: IMPALA-9092: Add support for creating external Kudu table ...................................................................... IMPALA-9092: Add support for creating external Kudu table In HMS-3 the translation layer converts a managed kudu table into a external kudu table and adds additional table property 'external.table.purge' to 'true'. This means any installation which is using HMS-3 (or a Hive version which has HIVE-22158) will always create Kudu tables as external tables. This is problematic since the the output of show create table will now be different and may confuse the users. In order to improve the user experience of such synchronized tables (external tables with external.table.purge property set to true), this patch adds support in Impala to create external Kudu tables. Previous versions of Impala disallowed creating a external Kudu table if the Kudu table did not exist. After this patch, Impala will check if the Kudu table exists and if it does not it will create a Kudu table based on the schema provided in the create table statement. However, this applies to only the synchronized tables. Previous way to create a pure external table behaves the same. Following syntax of creating a synchronized table is now allowed: CREATE EXTERNAL TABLE foo ( id int PRIMARY KEY, name string) PARTITION BY HASH PARTITIONS 8 STORED AS KUDU TBLPROPERTIES ('external.table.purge'='true') The syntax is very similar to creating a managed table, except for the EXTERNAL keyword and additional table property. A synchronized table will behave similar to managed Kudu tables (drops and renames are allowed). The output of show create table on a synchronized table will display the full column and partition spec similar to the managed tables. Testing: 1. After the CDP version bump all of the existing Kudu tables now create synchronized tables so there is good coverage there. 2. Added additional tests which create synchronized tables and compares the show create table output. Change-Id: I76f81d41db0cf2269ee1b365857164a43677e14d --- M fe/src/main/java/org/apache/impala/analysis/AlterTableSetTblProperties.java M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java M fe/src/main/java/org/apache/impala/analysis/ToSqlUtils.java M fe/src/main/java/org/apache/impala/catalog/KuduTable.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java M fe/src/test/java/org/apache/impala/catalog/CatalogTest.java M fe/src/test/java/org/apache/impala/catalog/local/LocalCatalogTest.java M fe/src/test/java/org/apache/impala/common/FrontendFixture.java M testdata/workloads/functional-query/queries/QueryTest/show-create-table.test M tests/common/skip.py M tests/metadata/test_show_create_table.py M tests/query_test/test_kudu.py 15 files changed, 703 insertions(+), 256 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/50/14750/1 -- To view, visit http://gerrit.cloudera.org:8080/14750 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I76f81d41db0cf2269ee1b365857164a43677e14d Gerrit-Change-Number: 14750 Gerrit-PatchSet: 1 Gerrit-Owner: Vihang Karajgaonkar <vih...@cloudera.com>