[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Thomas Marshall has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13375 ) Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration This commit supports the actual handling of CREATE TABLE DDL for managed Kudu tables when integration with Hive Metastore is enabled. When Kudu/HMS integration is enabled, for CREATE TABLE statement, Impala can rely on Kudu to create the table in the HMS. Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Reviewed-on: http://gerrit.cloudera.org:8080/13375 Reviewed-by: Thomas Marshall Reviewed-by: Grant Henke Tested-by: Thomas Marshall --- M fe/src/main/java/org/apache/impala/catalog/KuduTable.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java M tests/common/custom_cluster_test_suite.py M tests/common/kudu_test_suite.py M tests/common/skip.py M tests/custom_cluster/test_kudu.py M tests/query_test/test_kudu.py 10 files changed, 306 insertions(+), 20 deletions(-) Approvals: Thomas Marshall: Looks good to me, approved; Verified Grant Henke: Looks good to me, but someone else must approve -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 10 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/13375 ) Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 9: Verified+1 There haven't been any code changes since the verify job on patch set 7, so I'll go ahead and verify it (the more recent verify failure was because I aborted the job) -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 9 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Tue, 04 Jun 2019 17:36:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13375 ) Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 9: -Code-Review Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/3500/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 9 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Tue, 04 Jun 2019 17:33:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/13375 ) Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 9: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 9 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Tue, 04 Jun 2019 17:33:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/13375 ) Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 8: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 8 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Tue, 04 Jun 2019 17:31:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Hello Thomas Marshall, Alexey Serbin, Andrew Wong, Grant Henke, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13375 to look at the new patch set (#9). Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration This commit supports the actual handling of CREATE TABLE DDL for managed Kudu tables when integration with Hive Metastore is enabled. When Kudu/HMS integration is enabled, for CREATE TABLE statement, Impala can rely on Kudu to create the table in the HMS. Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 --- M fe/src/main/java/org/apache/impala/catalog/KuduTable.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java M tests/common/custom_cluster_test_suite.py M tests/common/kudu_test_suite.py M tests/common/skip.py M tests/custom_cluster/test_kudu.py M tests/query_test/test_kudu.py 10 files changed, 306 insertions(+), 20 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/75/13375/9 -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 9 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13375 ) Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 8: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 8 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Tue, 04 Jun 2019 17:30:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13375 ) Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 8: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4403/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 8 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Tue, 04 Jun 2019 17:31:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Alexey Serbin has posted comments on this change. (
http://gerrit.cloudera.org:8080/13375 )
Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with
Kudu/HMS integration
..
Patch Set 7:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java
File fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java:
http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java@120
PS4, Line 120:* Return the value of thrift URI for the remote Hive
Metastore.
:*/
: public static String
getHiveMetastoreUrisKeyValue(IMetaStoreClient client)
: throws ConfigValSecurityException, TException {
: return client.getConfigValue(
: HIVE_METASTORE_URIS_KEY, DEFAULT_HIVE_METASTORE_URIS);
: }
:
> This is to ensure always use the correct key (HIVE_METASTORE_URIS_KEY) when
If the constant HIVE_METASTORE_URIS_KEY is used, then I don't see much risk
specifying a wrong key accidentally. However, if I'm OK with keeping it since
it was just a nit comment, actually.
--
To view, visit http://gerrit.cloudera.org:8080/13375
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502
Gerrit-Change-Number: 13375
Gerrit-PatchSet: 7
Gerrit-Owner: Hao Hao
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Hao Hao
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Thomas Marshall
Gerrit-Comment-Date: Tue, 04 Jun 2019 16:50:48 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/13375 ) Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 7: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 7 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Tue, 04 Jun 2019 12:53:38 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13375 ) Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 6: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 6 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Tue, 04 Jun 2019 04:05:38 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13375 ) Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 7: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3490/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 7 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Tue, 04 Jun 2019 00:15:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Hello Thomas Marshall, Alexey Serbin, Andrew Wong, Grant Henke, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13375 to look at the new patch set (#7). Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration This commit supports the actual handling of CREATE TABLE DDL for managed Kudu tables when integration with Hive Metastore is enabled. When Kudu/HMS integration is enabled, for CREATE TABLE statement, Impala can rely on Kudu to create the table in the HMS. Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 --- M fe/src/main/java/org/apache/impala/catalog/KuduTable.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java M tests/common/custom_cluster_test_suite.py M tests/common/kudu_test_suite.py M tests/common/skip.py M tests/custom_cluster/test_kudu.py M tests/query_test/test_kudu.py 10 files changed, 306 insertions(+), 20 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/75/13375/7 -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 7 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/13375 ) Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 6: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 6 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Mon, 03 Jun 2019 22:42:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13375 ) Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 6: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4394/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 6 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Mon, 03 Jun 2019 22:23:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/13375 ) Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 6: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 6 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Mon, 03 Jun 2019 22:22:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13375 ) Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3488/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 6 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Mon, 03 Jun 2019 20:37:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Hello Thomas Marshall, Alexey Serbin, Andrew Wong, Grant Henke, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13375 to look at the new patch set (#6). Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration This commit supports the actual handling of CREATE TABLE DDL for managed Kudu tables when integration with Hive Metastore is enabled. When Kudu/HMS integration is enabled, for CREATE TABLE statement, Impala can rely on Kudu to create the table in the HMS. Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 --- M fe/src/main/java/org/apache/impala/catalog/KuduTable.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java M tests/common/custom_cluster_test_suite.py M tests/common/kudu_test_suite.py M tests/common/skip.py M tests/custom_cluster/test_kudu.py M tests/query_test/test_kudu.py 10 files changed, 306 insertions(+), 20 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/75/13375/6 -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 6 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Hao Hao has posted comments on this change. ( http://gerrit.cloudera.org:8080/13375 ) Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/13375/5/tests/query_test/test_kudu.py File tests/query_test/test_kudu.py: http://gerrit.cloudera.org:8080/#/c/13375/5/tests/query_test/test_kudu.py@1130 PS5, Line 1130: @SkipIfKudu.hms_integration_enabled > I still think that a lot of the tests in this file and elsewhere that have The tests that are skipped when HMS integration enabled for mainly two reasons: 1) the table name has "impala::" prefix hardcoded. 2) using external table, which could has name collision with managed table created by Kudu. Yeah, I think it makes sense to parameterize some tests so that they all run both with and without hms integration enabled. (I was trying to do it, however don't find a good way to achieve it yet). I can file a JIRA to track this. Non HMS integrated mode is not something going to be aggressively deprecated. So we are planning to maintain both mode for awhile. -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 5 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Mon, 03 Jun 2019 19:15:47 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/13375 ) Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/13375/5/tests/query_test/test_kudu.py File tests/query_test/test_kudu.py: http://gerrit.cloudera.org:8080/#/c/13375/5/tests/query_test/test_kudu.py@1130 PS5, Line 1130: @SkipIfKudu.hms_integration_enabled I still think that a lot of the tests in this file and elsewhere that have been skipped don't need to be - eg. I'm not sure why this one wouldn't work as is with hms integration enabled. It also occurs to me - as written, this 'skip' won't ever actually be triggered in Impala's normal test runs, as we'll always run these tests against the default minicluster config, which doesn't enable HMS integration. Of course, even in that case people may use these tests to run in other situations, eg. there have been efforts to run them against real clusters, so it still makes sense to include the 'skip' for any tests that genuinely won't ever work with hms integration turned on. I'm wondering though if it would be worth the effort and testing time to parameterize these tests so that they all run both with and without hms integration enabled, eg. by having custom_cluster/TestKuduHMSIntegration inherent from some of these classes? A lot of the tests in this file really won't be impacted by the integration (eg. the scan tests) and it would be basically wasted test time to run them in both configurations, but a lot of them are affected (eg. this test is one I would definitely want to know works with HMS integration turned on) One question I have is what's Kudu plan for how this evolves in future version - i.e. is the non-HMS integrated functionality going to be aggressively deprecated, such that Impala will probably want to just make hms integration turned on the default config in our minicluster, or is the plan to maintain both code paths for awhile? Anyways, I think this patch has fairly good coverage as is, and we can probably just file a JIRA to clean some of this up later. -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 5 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Mon, 03 Jun 2019 18:09:57 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13375 ) Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 5 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Mon, 03 Jun 2019 03:24:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13375 ) Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4387/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 5 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Sun, 02 Jun 2019 21:52:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13375 ) Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3478/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 5 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Sun, 02 Jun 2019 21:36:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13375 ) Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/13375/5/tests/custom_cluster/test_kudu.py File tests/custom_cluster/test_kudu.py: http://gerrit.cloudera.org:8080/#/c/13375/5/tests/custom_cluster/test_kudu.py@191 PS5, Line 191: flake8: W292 no newline at end of file -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 5 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Sun, 02 Jun 2019 20:58:53 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Hello Thomas Marshall, Alexey Serbin, Andrew Wong, Grant Henke, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13375 to look at the new patch set (#5). Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration This commit supports the actual handling of CREATE TABLE DDL for managed Kudu tables when integration with Hive Metastore is enabled. When Kudu/HMS integration is enabled, for CREATE TABLE statement, Impala can rely on Kudu to create the table in the HMS. Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 --- M fe/src/main/java/org/apache/impala/catalog/KuduTable.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java M tests/common/custom_cluster_test_suite.py M tests/common/kudu_test_suite.py M tests/common/skip.py M tests/custom_cluster/test_kudu.py M tests/query_test/test_kudu.py 10 files changed, 305 insertions(+), 20 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/75/13375/5 -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 5 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Hao Hao has posted comments on this change. (
http://gerrit.cloudera.org:8080/13375 )
Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with
Kudu/HMS integration
..
Patch Set 4:
(9 comments)
http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/catalog/KuduTable.java
File fe/src/main/java/org/apache/impala/catalog/KuduTable.java:
http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/catalog/KuduTable.java@172
PS4, Line 172: org.apache.hadoop.hive.metastore.api.Table msTbl,
: boolean isHMSIntegrationEnabled
> these can fit on the same line
It exceeds the line length limit (91).
http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/catalog/KuduTable.java@231
PS4, Line 231: if (hmsHosts != null && kuduHmsHosts != null
&(kuduHmsHosts)) {
> nit: separate by a space
Done
http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/catalog/KuduTable.java@234
PS4, Line 234: throw new ImpalaRuntimeException(
: String.format("Kudu is integrated with a different
Hive Metastore " +
: "than that used by Impala, Kudu is configured to
use the HMS: " +
: "%s, while Impala is configured to use the HMS:
%s",
: kuduHmsUris, hmsUris));
> readability nit: remove 'else' and move this out of the scope if/else scope
Done
http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java:
http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1930
PS4, Line 1930: !KuduTable.isHMSIntegrationEnabledAndValidate(masterHosts,
:
hmsUris
> formatting - put everything after the '=' on the next line, indented by 4
Done
http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java
File fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java:
http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java@87
PS4, Line 87: public static final String HIVE_METASTORE_URIS_KEY =
: "hive.metastore.uris";
> This can fit on 1 line
Done
http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java@120
PS4, Line 120: /**
:* Return the value of thrift URI for the remote Hive
Metastore.
:*/
: public static String
getHiveMetastoreUrisKeyValue(IMetaStoreClient client)
: throws ConfigValSecurityException, TException {
: return client.getConfigValue(
: HIVE_METASTORE_URIS_KEY, DEFAULT_HIVE_METASTORE_URIS);
: }
> Why to create this shortcut instead of just using getMetastoreConfigValue()
This is to ensure always use the correct key (HIVE_METASTORE_URIS_KEY) when
calling this function.
http://gerrit.cloudera.org:8080/#/c/13375/4/tests/custom_cluster/test_kudu.py
File tests/custom_cluster/test_kudu.py:
http://gerrit.cloudera.org:8080/#/c/13375/4/tests/custom_cluster/test_kudu.py@129
PS4, Line 129: manged
> typo
Done
http://gerrit.cloudera.org:8080/#/c/13375/4/tests/query_test/test_kudu.py
File tests/query_test/test_kudu.py:
http://gerrit.cloudera.org:8080/#/c/13375/4/tests/query_test/test_kudu.py@93
PS4, Line 93: @SkipIfKudu.hms_integration_enabled
> Why would a test like this, which is focused on scanning kudu tables, need
Done
http://gerrit.cloudera.org:8080/#/c/13375/4/tests/query_test/test_kudu.py@175
PS4, Line 175: @SkipIfKudu.hms_integration_enabled
> I assume that for cases like these, where the table is modified outside of
I am not sure it is the case, as far as my understanding of IMPALA-4828 is due
to the table schema is cached in impala which is different from the one in
Kudu. While with HMS integration, Kudu only automatically update the schema in
the HMS. Therefore, I don't see the behavior will change. Or I am missing
something?
--
To view, visit http://gerrit.cloudera.org:8080/13375
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502
Gerrit-Change-Number: 13375
Gerrit-PatchSet: 4
Gerrit-Owner: Hao Hao
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Hao Hao
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Thomas Marshall
Gerrit-Comment-Date: Sun, 02 Jun 2019 20:57:00 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/13375 ) Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 4: (6 comments) http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/catalog/KuduTable.java File fe/src/main/java/org/apache/impala/catalog/KuduTable.java: http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/catalog/KuduTable.java@172 PS4, Line 172: org.apache.hadoop.hive.metastore.api.Table msTbl, : boolean isHMSIntegrationEnabled these can fit on the same line http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1930 PS4, Line 1930: !KuduTable.isHMSIntegrationEnabledAndValidate(masterHosts, : hmsUris formatting - put everything after the '=' on the next line, indented by 4 http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java File fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java: http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java@87 PS4, Line 87: public static final String HIVE_METASTORE_URIS_KEY = : "hive.metastore.uris"; This can fit on 1 line http://gerrit.cloudera.org:8080/#/c/13375/4/tests/custom_cluster/test_kudu.py File tests/custom_cluster/test_kudu.py: http://gerrit.cloudera.org:8080/#/c/13375/4/tests/custom_cluster/test_kudu.py@129 PS4, Line 129: manged typo http://gerrit.cloudera.org:8080/#/c/13375/4/tests/query_test/test_kudu.py File tests/query_test/test_kudu.py: http://gerrit.cloudera.org:8080/#/c/13375/4/tests/query_test/test_kudu.py@93 PS4, Line 93: @SkipIfKudu.hms_integration_enabled Why would a test like this, which is focused on scanning kudu tables, need to be skipped if hms integration is turned on? http://gerrit.cloudera.org:8080/#/c/13375/4/tests/query_test/test_kudu.py@175 PS4, Line 175: @SkipIfKudu.hms_integration_enabled I assume that for cases like these, where the table is modified outside of Impala, we expect that Impala will automatically pick up the changes and the error won't occur? -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 4 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 31 May 2019 23:38:25 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Alexey Serbin has posted comments on this change. (
http://gerrit.cloudera.org:8080/13375 )
Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with
Kudu/HMS integration
..
Patch Set 4:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java
File fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java:
http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java@120
PS4, Line 120: /**
:* Return the value of thrift URI for the remote Hive
Metastore.
:*/
: public static String
getHiveMetastoreUrisKeyValue(IMetaStoreClient client)
: throws ConfigValSecurityException, TException {
: return client.getConfigValue(
: HIVE_METASTORE_URIS_KEY, DEFAULT_HIVE_METASTORE_URIS);
: }
Why to create this shortcut instead of just using getMetastoreConfigValue()
below specifying the HIVE_METASTORE_URIS_KEY ?
--
To view, visit http://gerrit.cloudera.org:8080/13375
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502
Gerrit-Change-Number: 13375
Gerrit-PatchSet: 4
Gerrit-Owner: Hao Hao
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Hao Hao
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Thomas Marshall
Gerrit-Comment-Date: Fri, 31 May 2019 23:12:19 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Alexey Serbin has posted comments on this change. (
http://gerrit.cloudera.org:8080/13375 )
Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with
Kudu/HMS integration
..
Patch Set 4:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/catalog/KuduTable.java
File fe/src/main/java/org/apache/impala/catalog/KuduTable.java:
http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/catalog/KuduTable.java@231
PS4, Line 231: if (hmsHosts != null && kuduHmsHosts != null
&(kuduHmsHosts)) {
nit: separate by a space
http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/catalog/KuduTable.java@234
PS4, Line 234: throw new ImpalaRuntimeException(
: String.format("Kudu is integrated with a different
Hive Metastore " +
: "than that used by Impala, Kudu is configured to
use the HMS: " +
: "%s, while Impala is configured to use the HMS:
%s",
: kuduHmsUris, hmsUris));
readability nit: remove 'else' and move this out of the scope if/else scope.
--
To view, visit http://gerrit.cloudera.org:8080/13375
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502
Gerrit-Change-Number: 13375
Gerrit-PatchSet: 4
Gerrit-Owner: Hao Hao
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Hao Hao
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Thomas Marshall
Gerrit-Comment-Date: Fri, 31 May 2019 21:46:10 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13375 ) Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3433/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 4 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Thu, 30 May 2019 05:29:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Hello Thomas Marshall, Alexey Serbin, Andrew Wong, Grant Henke, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13375 to look at the new patch set (#4). Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration This commit supports the actual handling of CREATE TABLE DDL for managed Kudu tables when integration with Hive Metastore is enabled. When Kudu/HMS integration is enabled, for CREATE TABLE statement, Impala can rely on Kudu to create the table in the HMS. Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 --- M fe/src/main/java/org/apache/impala/catalog/KuduTable.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java M fe/src/test/resources/hive-site.xml.py M testdata/bin/run-hive-server.sh M tests/common/custom_cluster_test_suite.py M tests/common/kudu_test_suite.py M tests/common/skip.py M tests/custom_cluster/test_kudu.py M tests/query_test/test_kudu.py 12 files changed, 266 insertions(+), 21 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/75/13375/4 -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 4 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Alexey Serbin has posted comments on this change. (
http://gerrit.cloudera.org:8080/13375 )
Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with
Kudu/HMS integration
..
Patch Set 3:
(7 comments)
did a quick first pass
http://gerrit.cloudera.org:8080/#/c/13375/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/13375/3//COMMIT_MSG@10
PS3, Line 10: with Kudu's integration with the Hive Metastore
nit: when integration with Hive Metastore is enabled
http://gerrit.cloudera.org:8080/#/c/13375/3//COMMIT_MSG@11
PS3, Line 11: after the table is created in the
: Kudu, relies on Kudu to create the table in the HMS.
nit: consider rewriting it to be easier to parse
http://gerrit.cloudera.org:8080/#/c/13375/3/fe/src/main/java/org/apache/impala/catalog/KuduTable.java
File fe/src/main/java/org/apache/impala/catalog/KuduTable.java:
http://gerrit.cloudera.org:8080/#/c/13375/3/fe/src/main/java/org/apache/impala/catalog/KuduTable.java@117
PS3, Line 117: if (kuduTableName_ == null || kuduTableName_.isEmpty()) {
: // When 'kudu.table_name' property is empty, it implies
Kudu/HMS
: // integration is enabled.
: // TODO: remove this hack once Kudu support
'kudu.table_name'
: // property with the new storage handler.
Would it make sense to create a method/function that will hide these
implementation details, so once the new storage handler is available, you will
need to update it in one place only.
http://gerrit.cloudera.org:8080/#/c/13375/3/fe/src/main/java/org/apache/impala/catalog/KuduTable.java@260
PS3, Line 260: if (kuduTableName_ == null || kuduTableName_.isEmpty()) {
: // When 'kudu.table_name' property is empty, it implies
Kudu/HMS
: // integration is enabled.
: // TODO: remove this hack once Kudu support
'kudu.table_name'
: // property with the new storage handler.
ditto -- would it help to create a function/method and then re-use it in such
cases.
http://gerrit.cloudera.org:8080/#/c/13375/3/tests/common/skip.py
File tests/common/skip.py:
http://gerrit.cloudera.org:8080/#/c/13375/3/tests/common/skip.py@110
PS3, Line 110: hms_integration
nit: maybe rename into hms_integration_enabled ?
http://gerrit.cloudera.org:8080/#/c/13375/3/tests/common/skip.py@112
PS3, Line 112: to be set in Kudu
to not be set ?
Maybe, replace the whole 'reason' string with
"Test assumes Kudu+HMS integration is not enabled."
http://gerrit.cloudera.org:8080/#/c/13375/3/tests/custom_cluster/test_kudu.py
File tests/custom_cluster/test_kudu.py:
http://gerrit.cloudera.org:8080/#/c/13375/3/tests/custom_cluster/test_kudu.py@128
PS3, Line 128: self.run_test_case('QueryTest/kudu_create', vector,
use_db=unique_database)
> What scenarios is this testing? Hard to tell without being more familiar wi
+1
--
To view, visit http://gerrit.cloudera.org:8080/13375
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502
Gerrit-Change-Number: 13375
Gerrit-PatchSet: 3
Gerrit-Owner: Hao Hao
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Hao Hao
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Thomas Marshall
Gerrit-Comment-Date: Tue, 28 May 2019 18:22:23 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Andrew Wong has posted comments on this change. (
http://gerrit.cloudera.org:8080/13375 )
Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with
Kudu/HMS integration
..
Patch Set 3:
(6 comments)
http://gerrit.cloudera.org:8080/#/c/13375/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java:
http://gerrit.cloudera.org:8080/#/c/13375/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1893
PS3, Line 1893:* 1. For managed tables, if Kudu's integration with the Hive
Metastore is not enabled,
:*the Kudu table is first created in the Kudu storage
engine, then in the HMS.
:*Failure to add the table in HMS results in the table
being dropped from Kudu.
:*If enabled, after the table is created in the Kudu,
relies on Kudu to create
:*the table in the HMS.
:*For external tables, only creates the tables in the HMS
(regardless Kudu's
:*integration with the Hive Metastore enabled or not).
:*
:* 2. And finally creates the table in the catalog cache.
nit: the structure of this felt a bit strange since there are three
non-partitioned cases to think about in #1. Perhaps make that distinction a bit
clearer as (+ some minor grammar tweaks):
For managed tables:
- If Kudu's integration with the Hive Metastore is not enabled, the Kudu table
is first created in Kudu, then in the HMS.
- Otherwise, when the table is created in Kudu, we rely on Kudu to have created
the table in the HMS.
For external tables:
- We only create the table in the HMS (regardless of Kudu's integration with
the Hive Metastore).
After the above is complete, we create the table in the catalog cache.
It'd also be nice if the code followed this structure.
http://gerrit.cloudera.org:8080/#/c/13375/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1909
PS3, Line 1909: // Check if Kudu's integration with the Hive Metastore is
enabled, and validate
: // the configuration.
: String masterHosts =
newTable.getParameters().get(KuduTable.KEY_MASTER_HOSTS);
: String hmsUris;
: try {
: hmsUris = MetaStoreUtil.getHiveMetastoreUrisKeyValue(
: catalog_.getMetaStoreClient().getHiveClient());
: } catch (Exception e) {
: throw new RuntimeException(String.format("Failed to get
the Hive Metastore " +
: "configuration for table '%s' ",
newTable.getTableName()), e);
: }
: boolean isHMSIntegrationEnabled =
:
KuduTable.isHMSIntegrationEnabledAndValidate(masterHosts, hmsUris);
Shouldn't be not be checking this in the case of an external table?
http://gerrit.cloudera.org:8080/#/c/13375/3/fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java
File fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java:
http://gerrit.cloudera.org:8080/#/c/13375/3/fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java@99
PS3, Line 99: (Validation of Kudu is configured to the same
: // Hive Metastore as Impala should have been done before
this).
It's not clear where this is happening. Might be worth noting.
Alternatively take this sentence out; not sure how useful it is. And presumably
we'll have more of these calls in other DDL, so not worth repeating everywhere.
http://gerrit.cloudera.org:8080/#/c/13375/3/fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java@283
PS3, Line 283: // External table should not have table ID.
Not your fault, but it isn't quite obvious that this is an external table.
Could you add a precheck for that here? And maybe update the comment explaining
that this should only be used for external tables?
http://gerrit.cloudera.org:8080/#/c/13375/3/tests/custom_cluster/test_kudu.py
File tests/custom_cluster/test_kudu.py:
http://gerrit.cloudera.org:8080/#/c/13375/3/tests/custom_cluster/test_kudu.py@116
PS3, Line 116: class TestKuduHMSIntegration(CustomClusterTestSuite,
KuduTestSuite):
Could you add a brief description of this test, similar to L99?
http://gerrit.cloudera.org:8080/#/c/13375/3/tests/custom_cluster/test_kudu.py@128
PS3, Line 128: self.run_test_case('QueryTest/kudu_create', vector,
use_db=unique_database)
What scenarios is this testing? Hard to tell without being more familiar with
the test fixtures. Maybe add a comment?
Does this cover managed _and_ external tables?
--
To view, visit http://gerrit.cloudera.org:8080/13375
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13375 ) Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3349/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 3 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Thu, 23 May 2019 06:09:31 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13375 ) Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 3: (2 comments) http://gerrit.cloudera.org:8080/#/c/13375/3/tests/custom_cluster/test_kudu.py File tests/custom_cluster/test_kudu.py: http://gerrit.cloudera.org:8080/#/c/13375/3/tests/custom_cluster/test_kudu.py@116 PS3, Line 116: class TestKuduHMSIntegration(CustomClusterTestSuite, KuduTestSuite): flake8: E302 expected 2 blank lines, found 1 http://gerrit.cloudera.org:8080/#/c/13375/3/tests/custom_cluster/test_kudu.py@129 PS3, Line 129: flake8: W391 blank line at end of file -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 3 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 23 May 2019 04:50:57 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Hello Grant Henke, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13375 to look at the new patch set (#3). Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration This commit supports the actual handling of CREATE TABLE DDL for managed Kudu tables with Kudu's integration with the Hive Metastore. When Kudu/HMS integration is enabled, after the table is created in the Kudu, relies on Kudu to create the table in the HMS. Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 --- M fe/src/main/java/org/apache/impala/catalog/KuduTable.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java M fe/src/test/resources/hive-site.xml.py M testdata/bin/run-hive-server.sh M tests/common/custom_cluster_test_suite.py M tests/common/kudu_test_suite.py M tests/common/skip.py M tests/custom_cluster/test_kudu.py M tests/query_test/test_kudu.py 12 files changed, 183 insertions(+), 15 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/75/13375/3 -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 3 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/13375 ) Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/13375/1/fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/13375/1/fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java@102 PS1, Line 102: String tableId = table.getTableId(); > I think we should avoid that, as without HMS integration enabled, it is har True, sounds good. -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 1 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 22 May 2019 13:00:55 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13375 ) Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 2: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/3320/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 2 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 22 May 2019 03:47:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Impala Public Jenkins has posted comments on this change. (
http://gerrit.cloudera.org:8080/13375 )
Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with
Kudu/HMS integration
..
Patch Set 2:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/13375/2/fe/src/main/java/org/apache/impala/catalog/KuduTable.java
File fe/src/main/java/org/apache/impala/catalog/KuduTable.java:
http://gerrit.cloudera.org:8080/#/c/13375/2/fe/src/main/java/org/apache/impala/catalog/KuduTable.java@212
PS2, Line 212: private static List parseUris(String uris, String
defaultPort) throws URISyntaxException {
line too long (97 > 90)
http://gerrit.cloudera.org:8080/#/c/13375/2/testdata/bin/run-hive-server.sh
File testdata/bin/run-hive-server.sh:
http://gerrit.cloudera.org:8080/#/c/13375/2/testdata/bin/run-hive-server.sh@87
PS2, Line 87: #
JAR_LOCATION="$MVN_REPO/org/apache/kudu/kudu-hive/$KUDU_VERSION/kudu-hive-$KUDU_VERSION.jar"
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/13375/2/tests/custom_cluster/test_kudu.py
File tests/custom_cluster/test_kudu.py:
http://gerrit.cloudera.org:8080/#/c/13375/2/tests/custom_cluster/test_kudu.py@116
PS2, Line 116: class TestKuduHMSIntegration(CustomClusterTestSuite,
KuduTestSuite):
flake8: E302 expected 2 blank lines, found 1
--
To view, visit http://gerrit.cloudera.org:8080/13375
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502
Gerrit-Change-Number: 13375
Gerrit-PatchSet: 2
Gerrit-Owner: Hao Hao
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Wed, 22 May 2019 02:51:21 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Hao Hao has posted comments on this change. ( http://gerrit.cloudera.org:8080/13375 ) Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/13375/1/fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/13375/1/fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java@102 PS1, Line 102: String tableId = table.getTableId(); > You could probably always set the table id for managed tables, even if HMS I think we should avoid that, as without HMS integration enabled, it is hard to tell if the table id is still correct. -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 2 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 22 May 2019 02:51:45 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Hello Grant Henke, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13375 to look at the new patch set (#2). Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration This commit supports the actual handling of CREATE TABLE DDL for managed Kudu tables with Kudu's integration with the Hive Metastore. When Kudu/HMS integration is enabled, after the table is created in the Kudu, relies on Kudu to create the table in the HMS. Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 --- M fe/src/main/java/org/apache/impala/catalog/KuduTable.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java M fe/src/test/resources/hive-site.xml.py M testdata/bin/run-hive-server.sh M tests/common/custom_cluster_test_suite.py M tests/common/kudu_test_suite.py M tests/common/skip.py M tests/custom_cluster/test_kudu.py M tests/query_test/test_kudu.py 11 files changed, 205 insertions(+), 13 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/75/13375/2 -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 2 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Impala Public Jenkins
