Hao Hao has uploaded a new patch set (#10) to the change originally created by Vihang Karajgaonkar. ( http://gerrit.cloudera.org:8080/14397 )
Change subject: IMPALA-9030: Handle translated external Kudu tables ...................................................................... IMPALA-9030: Handle translated external Kudu tables In HMS 3.1 there is a default tranformer introduced which checks the client capabilities and transforms a table before creating it. Additionally, it also makes sure that any managed table which is created is transactional. If a user creates a managed table which is not transactional, it automatically converts such table as external and sets certain table properties to mark such transformed tables. This presents a problem for managed Kudu tables in Impala since managed and external tables are handled differently in Kudu. Specifically, if a Kudu table is managed, certain operations like drop table, rename table, alter table are performed on the Kudu side along with updating the catalog. If the Kudu table is external, the Kudu operations are skipped and only catalog side operations are performed. When the user creates a managed Kudu table, user expects that drop table, rename table should be updated by Impala automatically in Kudu as well. But since HMS 3 transforms such managed tables into external, currently Impala does not perform the Kudu side operations breaking the semantics for the user. This patch makes changes to Catalog so that it can detect such transformed external tables and perform Kudu side operations similar to what it was doing for managed Kudu table when talking with previous HMS versions. Note that this change is in preparation of bumping up the CDP build which will be done in a separate change. For the current CDP build number the patch is essentially a no-op. Testing: 1. Bumped up the CDP build number in a private build so that the HMS translation logic is pulled in. Ran all the tests. Without the patch there are many Kudu tests which were failing. After the patch none of the Kudu tests fail. There were additional Ranger tests which failed due to the CDP bump but those were unrelated to this patch and should be fixed as part of a separate change when the CDP build number is bumped up. Change-Id: I324523361c923b7d291cb4d0f1028b1a5b653b36 --- M fe/src/main/java/org/apache/impala/analysis/AlterTableSetTblProperties.java M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java M fe/src/main/java/org/apache/impala/catalog/KuduTable.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java M fe/src/test/java/org/apache/impala/common/FrontendFixture.java 8 files changed, 105 insertions(+), 40 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/97/14397/10 -- To view, visit http://gerrit.cloudera.org:8080/14397 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I324523361c923b7d291cb4d0f1028b1a5b653b36 Gerrit-Change-Number: 14397 Gerrit-PatchSet: 10 Gerrit-Owner: Vihang Karajgaonkar <vih...@cloudera.com> Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com> Gerrit-Reviewer: Grant Henke <granthe...@apache.org> Gerrit-Reviewer: Greg Solovyev <gsolov...@cloudera.com> Gerrit-Reviewer: Hao Hao <hao....@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Thomas Tauber-Marshall <tmarsh...@cloudera.com> Gerrit-Reviewer: Vihang Karajgaonkar <vih...@cloudera.com>
