[Impala-ASF-CR] Revert "Revert "IMPALA-7074: Update OWNER privilege on CREATE, DROP, and SET OWNER""
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11466 ) Change subject: Revert "Revert "IMPALA-7074: Update OWNER privilege on CREATE, DROP, and SET OWNER"" .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/708/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11466 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4b9276c36bf96afccd7b8ff48803a30b47062c3d Gerrit-Change-Number: 11466 Gerrit-PatchSet: 2 Gerrit-Owner: Adam Holley Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 19 Sep 2018 05:56:45 + Gerrit-HasComments: No
[Impala-ASF-CR] Revert "Revert "IMPALA-7074: Update OWNER privilege on CREATE, DROP, and SET OWNER""
Adam Holley has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/11466 ) Change subject: Revert "Revert "IMPALA-7074: Update OWNER privilege on CREATE, DROP, and SET OWNER"" .. Revert "Revert "IMPALA-7074: Update OWNER privilege on CREATE, DROP, and SET OWNER"" The problem was caused by update in Hive with changed notifications. HIVE-15180 was added but was incomplete and resulted in the break. HIVE-17747 fixed the issue by properly creating the messages. Change-Id: I4b9276c36bf96afccd7b8ff48803a30b47062c3d --- M bin/create-test-configuration.sh M bin/impala-config.sh M common/thrift/JniCatalog.thrift M fe/src/main/java/org/apache/impala/analysis/AlterDbSetOwnerStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterTableOrViewSetOwnerStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterViewStmt.java M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/CreateDbStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateOrAlterViewStmtBase.java M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateViewStmt.java M fe/src/main/java/org/apache/impala/analysis/DropDbStmt.java M fe/src/main/java/org/apache/impala/analysis/DropTableOrViewStmt.java M fe/src/main/java/org/apache/impala/analysis/GrantRevokePrivStmt.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/main/java/org/apache/impala/util/SentryPolicyService.java M fe/src/main/java/org/apache/impala/util/SentryProxy.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/testutil/CatalogServiceTestCatalog.java M fe/src/test/resources/mysql-hive-site.xml.template M fe/src/test/resources/postgresql-hive-site.xml.template M fe/src/test/resources/sentry-site.xml.template A fe/src/test/resources/sentry-site_no_oo.xml.template A fe/src/test/resources/sentry-site_oo.xml.template A fe/src/test/resources/sentry-site_oo_nogrant.xml.template M testdata/bin/run-sentry-service.sh M tests/authorization/test_grant_revoke.py A tests/authorization/test_owner_privileges.py M tests/common/custom_cluster_test_suite.py M tests/common/impala_test_suite.py 34 files changed, 1,365 insertions(+), 154 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/66/11466/2 -- To view, visit http://gerrit.cloudera.org:8080/11466 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4b9276c36bf96afccd7b8ff48803a30b47062c3d Gerrit-Change-Number: 11466 Gerrit-PatchSet: 2 Gerrit-Owner: Adam Holley Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] Revert "Revert "IMPALA-7074: Update OWNER privilege on CREATE, DROP, and SET OWNER""
Adam Holley has posted comments on this change. ( http://gerrit.cloudera.org:8080/11466 ) Change subject: Revert "Revert "IMPALA-7074: Update OWNER privilege on CREATE, DROP, and SET OWNER"" .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/11466/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11466/1//COMMIT_MSG@9 PS1, Line 9: The problem was caused by update in Hive with changed notifications. > nit: I'm just passing by, but you might want to include a sentence on what Done -- To view, visit http://gerrit.cloudera.org:8080/11466 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4b9276c36bf96afccd7b8ff48803a30b47062c3d Gerrit-Change-Number: 11466 Gerrit-PatchSet: 2 Gerrit-Owner: Adam Holley Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 19 Sep 2018 05:22:18 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Revert "Revert "IMPALA-7074: Update OWNER privilege on CREATE, DROP, and SET OWNER""
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/11466 ) Change subject: Revert "Revert "IMPALA-7074: Update OWNER privilege on CREATE, DROP, and SET OWNER"" .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/11466/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11466/1//COMMIT_MSG@9 PS1, Line 9: The problem was caused by update in Hive with changed notifications. nit: I'm just passing by, but you might want to include a sentence on what the problem was, and how it was addressed. -- To view, visit http://gerrit.cloudera.org:8080/11466 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4b9276c36bf96afccd7b8ff48803a30b47062c3d Gerrit-Change-Number: 11466 Gerrit-PatchSet: 1 Gerrit-Owner: Adam Holley Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 19 Sep 2018 04:48:28 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7579: fix test query profile contains all events on S3
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11461 ) Change subject: IMPALA-7579: fix test_query_profile_contains_all_events on S3 .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/11461 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7c6e0899455dd4e12636b959fab4bde79f02fb95 Gerrit-Change-Number: 11461 Gerrit-PatchSet: 2 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 19 Sep 2018 03:59:14 + Gerrit-HasComments: No
[Impala-ASF-CR] Revert "Revert "IMPALA-7074: Update OWNER privilege on CREATE, DROP, and SET OWNER""
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11466 ) Change subject: Revert "Revert "IMPALA-7074: Update OWNER privilege on CREATE, DROP, and SET OWNER"" .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/707/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11466 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4b9276c36bf96afccd7b8ff48803a30b47062c3d Gerrit-Change-Number: 11466 Gerrit-PatchSet: 1 Gerrit-Owner: Adam Holley Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 19 Sep 2018 03:33:15 + Gerrit-HasComments: No
[Impala-ASF-CR] Revert "Revert "IMPALA-7074: Update OWNER privilege on CREATE, DROP, and SET OWNER""
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11466 ) Change subject: Revert "Revert "IMPALA-7074: Update OWNER privilege on CREATE, DROP, and SET OWNER"" .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3175/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/11466 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4b9276c36bf96afccd7b8ff48803a30b47062c3d Gerrit-Change-Number: 11466 Gerrit-PatchSet: 1 Gerrit-Owner: Adam Holley Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 19 Sep 2018 02:59:43 + Gerrit-HasComments: No
[Impala-ASF-CR] Revert "Revert "IMPALA-7074: Update OWNER privilege on CREATE, DROP, and SET OWNER""
Adam Holley has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11466 Change subject: Revert "Revert "IMPALA-7074: Update OWNER privilege on CREATE, DROP, and SET OWNER"" .. Revert "Revert "IMPALA-7074: Update OWNER privilege on CREATE, DROP, and SET OWNER"" The problem was caused by update in Hive with changed notifications. Change-Id: I4b9276c36bf96afccd7b8ff48803a30b47062c3d --- M bin/create-test-configuration.sh M bin/impala-config.sh M common/thrift/JniCatalog.thrift M fe/src/main/java/org/apache/impala/analysis/AlterDbSetOwnerStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterTableOrViewSetOwnerStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterViewStmt.java M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/CreateDbStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateOrAlterViewStmtBase.java M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateViewStmt.java M fe/src/main/java/org/apache/impala/analysis/DropDbStmt.java M fe/src/main/java/org/apache/impala/analysis/DropTableOrViewStmt.java M fe/src/main/java/org/apache/impala/analysis/GrantRevokePrivStmt.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/main/java/org/apache/impala/util/SentryPolicyService.java M fe/src/main/java/org/apache/impala/util/SentryProxy.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/testutil/CatalogServiceTestCatalog.java M fe/src/test/resources/mysql-hive-site.xml.template M fe/src/test/resources/postgresql-hive-site.xml.template M fe/src/test/resources/sentry-site.xml.template A fe/src/test/resources/sentry-site_no_oo.xml.template A fe/src/test/resources/sentry-site_oo.xml.template A fe/src/test/resources/sentry-site_oo_nogrant.xml.template M testdata/bin/run-sentry-service.sh M tests/authorization/test_grant_revoke.py A tests/authorization/test_owner_privileges.py M tests/common/custom_cluster_test_suite.py M tests/common/impala_test_suite.py 34 files changed, 1,365 insertions(+), 154 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/66/11466/1 -- To view, visit http://gerrit.cloudera.org:8080/11466 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I4b9276c36bf96afccd7b8ff48803a30b47062c3d Gerrit-Change-Number: 11466 Gerrit-PatchSet: 1 Gerrit-Owner: Adam Holley
[native-toolchain-CR] IMPALA-6772: Build ORC lib without libhdfspp and apply ORC-403
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/11442 ) Change subject: IMPALA-6772: Build ORC lib without libhdfspp and apply ORC-403 .. Patch Set 1: (1 comment) I'm so sorry that I missed your updates... Just updated the patch. After applying ORC-403, I'm able to run fuzz_scanner_tests for orc for more than 3 days without any errors. Since ORC-403 is just merged and I made some minor changes in the PR. I'll run the fuzz test again for several days. http://gerrit.cloudera.org:8080/#/c/11442/1/build.sh File build.sh: http://gerrit.cloudera.org:8080/#/c/11442/1/build.sh@37 PS1, Line 37: if [ $PACKAGE == 'ORC' ]; then > Can you revert this part? I agree the ergonomics of build.sh are bad - you Sure. I agree that this is not elegant and prone to introduce errors in the future. -- To view, visit http://gerrit.cloudera.org:8080/11442 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1491b558a7972c0f77b0bfeb6e9cc04fda1734e9 Gerrit-Change-Number: 11442 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 19 Sep 2018 02:44:13 + Gerrit-HasComments: Yes
[native-toolchain-CR] IMPALA-6772: Build ORC lib without libhdfspp and apply ORC-403
Hello Tim Armstrong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11442 to look at the new patch set (#2). Change subject: IMPALA-6772: Build ORC lib without libhdfspp and apply ORC-403 .. IMPALA-6772: Build ORC lib without libhdfspp and apply ORC-403 ORC lib supports reading files in HDFS since 1.5.0. It includes libhdfs++ of Hadoop2, which causes errors when links with Impala. This patch adds a cmake option to exclude libhdfs++. Also apply ORC-403 to introduce more checks inside the ORC lib. Change-Id: I1491b558a7972c0f77b0bfeb6e9cc04fda1734e9 --- M buildall.sh M source/orc/build.sh A source/orc/orc-1.5.2-patches/0002-ORC-403-C-Add-checks-to-avoid-invalid-offsets-in-Inp.patch 3 files changed, 190 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/42/11442/2 -- To view, visit http://gerrit.cloudera.org:8080/11442 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1491b558a7972c0f77b0bfeb6e9cc04fda1734e9 Gerrit-Change-Number: 11442 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7579: fix test query profile contains all events on S3
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11461 ) Change subject: IMPALA-7579: fix test_query_profile_contains_all_events on S3 .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/706/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11461 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7c6e0899455dd4e12636b959fab4bde79f02fb95 Gerrit-Change-Number: 11461 Gerrit-PatchSet: 2 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 19 Sep 2018 01:04:54 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7579: fix test query profile contains all events on S3
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11461 ) Change subject: IMPALA-7579: fix test_query_profile_contains_all_events on S3 .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3174/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/11461 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7c6e0899455dd4e12636b959fab4bde79f02fb95 Gerrit-Change-Number: 11461 Gerrit-PatchSet: 2 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 19 Sep 2018 00:34:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7579: fix test query profile contains all events on S3
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/11461 ) Change subject: IMPALA-7579: fix test_query_profile_contains_all_events on S3 .. Patch Set 1: (5 comments) http://gerrit.cloudera.org:8080/#/c/11461/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11461/1//COMMIT_MSG@7 PS1, Line 7: IMPALA-7579: use the generic filesystem_client to delete and create files in test_query_profile_contains_all_events. > nit: keep commit message lines to 72 characters (and don't wrap the title l Done http://gerrit.cloudera.org:8080/#/c/11461/1//COMMIT_MSG@7 PS1, Line 7: IMPALA-7579: use the generic filesystem_client to delete and create files in test_query_profile_contains_all_events. > make this line less verbose... something like "fix S3 file ops in test_quer Done http://gerrit.cloudera.org:8080/#/c/11461/1//COMMIT_MSG@11 PS1, Line 11: is > nit: it Done http://gerrit.cloudera.org:8080/#/c/11461/1//COMMIT_MSG@11 PS1, Line 11: > nit: get rid of extra ws Done http://gerrit.cloudera.org:8080/#/c/11461/1/tests/query_test/test_observability.py File tests/query_test/test_observability.py: http://gerrit.cloudera.org:8080/#/c/11461/1/tests/query_test/test_observability.py@224 PS1, Line 224: tmp > is this where the db is created? I thought it was under 'test-warehouse' Thanks for the suggestion -- To view, visit http://gerrit.cloudera.org:8080/11461 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7c6e0899455dd4e12636b959fab4bde79f02fb95 Gerrit-Change-Number: 11461 Gerrit-PatchSet: 1 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 19 Sep 2018 00:33:13 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7579: fix test query profile contains all events on S3
Andrew Sherman has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/11461 ) Change subject: IMPALA-7579: fix test_query_profile_contains_all_events on S3 .. IMPALA-7579: fix test_query_profile_contains_all_events on S3 This bug was introduced by IMPALA-6568 which added the test_query_profile_contains_all_events test. This test creates a file in the filesystem so that it can be used by 'load data inpath'. The test was using the hdfs_client object to do file system operations, but this client only works with hdfs. Switch to using the filesystem_client object which can work on other filesystems, including s3. Also create the file that will be moved by 'load data inpath' under the unique database directory; this means the file can be created without having to check if it exists already and must be deleted. Change-Id: I7c6e0899455dd4e12636b959fab4bde79f02fb95 --- M tests/query_test/test_observability.py 1 file changed, 2 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/61/11461/2 -- To view, visit http://gerrit.cloudera.org:8080/11461 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I7c6e0899455dd4e12636b959fab4bde79f02fb95 Gerrit-Change-Number: 11461 Gerrit-PatchSet: 2 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-7488: Fix hang in test cancellation
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11465 ) Change subject: IMPALA-7488: Fix hang in test_cancellation .. Patch Set 2: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/3173/ -- To view, visit http://gerrit.cloudera.org:8080/11465 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9c4b570604f7706712eb8e19b1ce69bf35cf15e2 Gerrit-Change-Number: 11465 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 19 Sep 2018 00:31:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7488: Fix hang in test cancellation
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11465 ) Change subject: IMPALA-7488: Fix hang in test_cancellation .. Patch Set 2: Code-Review+1 (1 comment) wow http://gerrit.cloudera.org:8080/#/c/11465/2/shell/impala_shell.py File shell/impala_shell.py: http://gerrit.cloudera.org:8080/#/c/11465/2/shell/impala_shell.py@a536 PS2, Line 536: :( -- To view, visit http://gerrit.cloudera.org:8080/11465 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9c4b570604f7706712eb8e19b1ce69bf35cf15e2 Gerrit-Change-Number: 11465 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 18 Sep 2018 23:15:28 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7488: Fix hang in test cancellation
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11465 ) Change subject: IMPALA-7488: Fix hang in test_cancellation .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11465 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9c4b570604f7706712eb8e19b1ce69bf35cf15e2 Gerrit-Change-Number: 11465 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 18 Sep 2018 22:56:16 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7488: Fix hang in test cancellation
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11465 ) Change subject: IMPALA-7488: Fix hang in test_cancellation .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3173/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11465 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9c4b570604f7706712eb8e19b1ce69bf35cf15e2 Gerrit-Change-Number: 11465 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 18 Sep 2018 22:56:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7488: Fix hang in test cancellation
Philip Zeyliger has posted comments on this change. ( http://gerrit.cloudera.org:8080/11465 ) Change subject: IMPALA-7488: Fix hang in test_cancellation .. Patch Set 1: Code-Review+2 Nice find! -- To view, visit http://gerrit.cloudera.org:8080/11465 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9c4b570604f7706712eb8e19b1ce69bf35cf15e2 Gerrit-Change-Number: 11465 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 18 Sep 2018 22:34:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7525: [DOCS] SHOW GRANT USER
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11454 ) Change subject: IMPALA-7525: [DOCS] SHOW GRANT USER .. Patch Set 2: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/77/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/11454 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iaa39275c595d73b2fba293f77bb21aa3985dfa38 Gerrit-Change-Number: 11454 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 18 Sep 2018 21:37:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7525: [DOCS] SHOW GRANT USER
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11454 ) Change subject: IMPALA-7525: [DOCS] SHOW GRANT USER .. Patch Set 2: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/77/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/11454 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iaa39275c595d73b2fba293f77bb21aa3985dfa38 Gerrit-Change-Number: 11454 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 18 Sep 2018 21:31:02 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7525: [DOCS] SHOW GRANT USER
Hello Fredy Wijaya, Impala Public Jenkins, Adam Holley, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11454 to look at the new patch set (#2). Change subject: IMPALA-7525: [DOCS] SHOW GRANT USER .. IMPALA-7525: [DOCS] SHOW GRANT USER Documented the new SHOW GRANT USER statements: SHOW GRANT USER user_name SHOW GRANT USER user_name ON SERVER SHOW GRANT USER user_name ON DATABASE database_name SHOW GRANT USER user_name ON TABLE table_name SHOW GRANT USER user_name ON URI uri Change-Id: Iaa39275c595d73b2fba293f77bb21aa3985dfa38 --- M docs/shared/impala_common.xml M docs/topics/impala_show.xml 2 files changed, 41 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/54/11454/2 -- To view, visit http://gerrit.cloudera.org:8080/11454 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iaa39275c595d73b2fba293f77bb21aa3985dfa38 Gerrit-Change-Number: 11454 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-7525: [DOCS] SHOW GRANT USER
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/11454 ) Change subject: IMPALA-7525: [DOCS] SHOW GRANT USER .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/11454/1/docs/topics/impala_show.xml File docs/topics/impala_show.xml: http://gerrit.cloudera.org:8080/#/c/11454/1/docs/topics/impala_show.xml@516 PS1, Line 516: and other users that have been granted the : specified role. > I'm not sure this is valid. What is the specified role? Additionally, the Done -- To view, visit http://gerrit.cloudera.org:8080/11454 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iaa39275c595d73b2fba293f77bb21aa3985dfa38 Gerrit-Change-Number: 11454 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 18 Sep 2018 21:30:05 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7488: Fix hang in test cancellation
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11465 ) Change subject: IMPALA-7488: Fix hang in test_cancellation .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/705/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11465 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9c4b570604f7706712eb8e19b1ce69bf35cf15e2 Gerrit-Change-Number: 11465 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Tue, 18 Sep 2018 20:55:54 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7488: Fix hang in test cancellation
Thomas Marshall has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11465 Change subject: IMPALA-7488: Fix hang in test_cancellation .. IMPALA-7488: Fix hang in test_cancellation test_cancellation runs a impala-shell process with a query specified then sends a SIGINT and confirms that the shell cancels the query and exits. The hang was happening because the shell's signal handler was incorrectly using the same Thirft connection when calling Close() as the main shell thread, which is not thread safe. Testing: - Ran test_cancellation in a loop 500 times. Previously the hang would repro about every 10 runs. Change-Id: I9c4b570604f7706712eb8e19b1ce69bf35cf15e2 --- M shell/impala_shell.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/65/11465/1 -- To view, visit http://gerrit.cloudera.org:8080/11465 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I9c4b570604f7706712eb8e19b1ce69bf35cf15e2 Gerrit-Change-Number: 11465 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall
[Impala-ASF-CR] IMPALA-7348: fix race in KuduUtil.getKuduClient
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11423 ) Change subject: IMPALA-7348: fix race in KuduUtil.getKuduClient .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/11423 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iec50e9b592593cea5418e7a074bdda065184086f Gerrit-Change-Number: 11423 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Tue, 18 Sep 2018 20:01:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7348: fix race in KuduUtil.getKuduClient
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11423 ) Change subject: IMPALA-7348: fix race in KuduUtil.getKuduClient .. IMPALA-7348: fix race in KuduUtil.getKuduClient We've seen a symptom in our builds where tests will fail due to missing cardinality info for Kudu tables. This is being caused by a NullPointerException during planning when running the 'compute stats' phase of data loading. The NullPointerException occurs because KuduUtil.getKuduClient is not thread safe and, depending on interleaving, may return NULL if there are two concurrent queries accessing Kudu tables right after cluster startup when the KuduClient map has not been populated yet. The solution is to make KuduUtil.getKuduClient thread safe by switching the KuduClient map to be a ConcurrentHashMap. Change-Id: Iec50e9b592593cea5418e7a074bdda065184086f Reviewed-on: http://gerrit.cloudera.org:8080/11423 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M fe/src/main/java/org/apache/impala/util/KuduUtil.java 1 file changed, 8 insertions(+), 5 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/11423 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Iec50e9b592593cea5418e7a074bdda065184086f Gerrit-Change-Number: 11423 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] Revert "IMPALA-7477: Batch-oriented query set construction"
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11462 ) Change subject: Revert "IMPALA-7477: Batch-oriented query set construction" .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/11462 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I533c12f1b4cfc5b4a372ba834913975b5c52c5a8 Gerrit-Change-Number: 11462 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 18 Sep 2018 19:53:50 + Gerrit-HasComments: No
[Impala-ASF-CR] Fix caps and grammar of backend startup profile key
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11451 ) Change subject: Fix caps and grammar of backend startup profile key .. Fix caps and grammar of backend startup profile key * Make caps consistent with other keys * Use "start up" (verb) instead of "startup" (noun) Change-Id: Ie7d38a81364a57de90a01f4f65a375dc2e403e18 Reviewed-on: http://gerrit.cloudera.org:8080/11451 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/runtime/coordinator.cc 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/11451 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ie7d38a81364a57de90a01f4f65a375dc2e403e18 Gerrit-Change-Number: 11451 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] Fix caps and grammar of backend startup profile key
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11451 ) Change subject: Fix caps and grammar of backend startup profile key .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/11451 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie7d38a81364a57de90a01f4f65a375dc2e403e18 Gerrit-Change-Number: 11451 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 18 Sep 2018 19:38:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7420: different error code for internal cancellation
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11464 ) Change subject: IMPALA-7420: different error code for internal cancellation .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/704/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11464 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If25d5b539d68981359e4d881cae7b08728ba2999 Gerrit-Change-Number: 11464 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 18 Sep 2018 19:27:02 + Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-7477: Batch-oriented query set construction"
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11462 ) Change subject: Revert "IMPALA-7477: Batch-oriented query set construction" .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/11462/2/be/src/service/hs2-util.h File be/src/service/hs2-util.h: http://gerrit.cloudera.org:8080/#/c/11462/2/be/src/service/hs2-util.h@23 PS2, Line 23: /// Utility methods for converting from Impala (either an Expr result or a TColumnValue) to line too long (91 > 90) -- To view, visit http://gerrit.cloudera.org:8080/11462 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I533c12f1b4cfc5b4a372ba834913975b5c52c5a8 Gerrit-Change-Number: 11462 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 18 Sep 2018 19:11:56 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7420: different error code for internal cancellation
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11464 ) Change subject: IMPALA-7420: different error code for internal cancellation .. Patch Set 2: Here's the redone version with new error code as discussed. -- To view, visit http://gerrit.cloudera.org:8080/11464 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If25d5b539d68981359e4d881cae7b08728ba2999 Gerrit-Change-Number: 11464 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 18 Sep 2018 18:53:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7420: different error code for internal cancellation
Tim Armstrong has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/11464 ) Change subject: IMPALA-7420: different error code for internal cancellation .. IMPALA-7420: different error code for internal cancellation I started by converting scan and spill-to-disk because the cancellation there is always meant to be internal to the scan and spill-to-disk subsystems. I updated all places that checked for TErrorCode::CANCELLED to treat CANCELLED_INTERNALLY the same. This is to aid triage and debugging of bugs like IMPALA-7418 where an "internal" cancellation leaks out into the query state. This will make it easier to determine if an internal cancellation somehow "leaked" out. Testing: Ran exhaustive tests. Change-Id: If25d5b539d68981359e4d881cae7b08728ba2999 --- M be/src/benchmarks/status-benchmark.cc M be/src/common/status.cc M be/src/common/status.h M be/src/exec/hdfs-orc-scanner.cc M be/src/exec/hdfs-parquet-scanner.cc M be/src/exec/hdfs-scan-node.cc M be/src/exec/hdfs-scanner.cc M be/src/exec/scanner-context.cc M be/src/runtime/io/disk-io-mgr-test.cc M be/src/runtime/io/request-context.cc M be/src/runtime/io/request-context.h M be/src/runtime/runtime-state.cc M be/src/runtime/tmp-file-mgr-test.cc M be/src/runtime/tmp-file-mgr.cc M common/thrift/generate_error_codes.py 15 files changed, 71 insertions(+), 52 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/64/11464/2 -- To view, visit http://gerrit.cloudera.org:8080/11464 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If25d5b539d68981359e4d881cae7b08728ba2999 Gerrit-Change-Number: 11464 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong
[Impala-ASF-CR] Revert "IMPALA-7477: Batch-oriented query set construction"
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11462 ) Change subject: Revert "IMPALA-7477: Batch-oriented query set construction" .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/703/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11462 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I533c12f1b4cfc5b4a372ba834913975b5c52c5a8 Gerrit-Change-Number: 11462 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 18 Sep 2018 18:41:54 + Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-7477: Batch-oriented query set construction"
Tim Armstrong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11462 Change subject: Revert "IMPALA-7477: Batch-oriented query set construction" .. Revert "IMPALA-7477: Batch-oriented query set construction" This has been implicated in some incorrect results involving nulls in HS2. This reverts commit b288a6af2eda9631b2bad91896ae4bfd2a3fdf30. Change-Id: I533c12f1b4cfc5b4a372ba834913975b5c52c5a8 --- M be/src/exec/plan-root-sink.cc M be/src/exec/plan-root-sink.h M be/src/service/hs2-util.cc M be/src/service/hs2-util.h M be/src/service/query-result-set.cc M be/src/service/query-result-set.h 6 files changed, 161 insertions(+), 320 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/62/11462/2 -- To view, visit http://gerrit.cloudera.org:8080/11462 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I533c12f1b4cfc5b4a372ba834913975b5c52c5a8 Gerrit-Change-Number: 11462 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7579: use the generic filesystem client to delete and create files in test query profile contains all events.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11461 ) Change subject: IMPALA-7579: use the generic filesystem_client to delete and create files in test_query_profile_contains_all_events. .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/702/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11461 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7c6e0899455dd4e12636b959fab4bde79f02fb95 Gerrit-Change-Number: 11461 Gerrit-PatchSet: 1 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Tue, 18 Sep 2018 17:20:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7579: use the generic filesystem client to delete and create files in test query profile contains all events.
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/11461 ) Change subject: IMPALA-7579: use the generic filesystem_client to delete and create files in test_query_profile_contains_all_events. .. Patch Set 1: (4 comments) http://gerrit.cloudera.org:8080/#/c/11461/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11461/1//COMMIT_MSG@7 PS1, Line 7: IMPALA-7579: use the generic filesystem_client to delete and create files in test_query_profile_contains_all_events. make this line less verbose... something like "fix S3 file ops in test_query_profile_contains_all_events" http://gerrit.cloudera.org:8080/#/c/11461/1//COMMIT_MSG@11 PS1, Line 11: nit: get rid of extra ws http://gerrit.cloudera.org:8080/#/c/11461/1//COMMIT_MSG@11 PS1, Line 11: is nit: it http://gerrit.cloudera.org:8080/#/c/11461/1/tests/query_test/test_observability.py File tests/query_test/test_observability.py: http://gerrit.cloudera.org:8080/#/c/11461/1/tests/query_test/test_observability.py@224 PS1, Line 224: tmp is this where the db is created? I thought it was under 'test-warehouse' if using the unique_database fixture, and you use the dir there to add a file, then the fixture should take care of removing that file. you can probably simplify here by removing L225. there might be other helper fns around to get the path given the db name. -- To view, visit http://gerrit.cloudera.org:8080/11461 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7c6e0899455dd4e12636b959fab4bde79f02fb95 Gerrit-Change-Number: 11461 Gerrit-PatchSet: 1 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Tue, 18 Sep 2018 16:51:28 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7579: use the generic filesystem client to delete and create files in test query profile contains all events.
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/11461 ) Change subject: IMPALA-7579: use the generic filesystem_client to delete and create files in test_query_profile_contains_all_events. .. Patch Set 1: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/11461/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11461/1//COMMIT_MSG@7 PS1, Line 7: IMPALA-7579: use the generic filesystem_client to delete and create files in test_query_profile_contains_all_events. nit: keep commit message lines to 72 characters (and don't wrap the title line, just make it shorter, eg. "use filesystem_client in test_query_profile_contains_all_events" or "fix test_query_profile_contains_all_events on S3") -- To view, visit http://gerrit.cloudera.org:8080/11461 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7c6e0899455dd4e12636b959fab4bde79f02fb95 Gerrit-Change-Number: 11461 Gerrit-PatchSet: 1 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Tue, 18 Sep 2018 16:42:20 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7527: add fetch-from-catalogd cache info to profile
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11388 ) Change subject: IMPALA-7527: add fetch-from-catalogd cache info to profile .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/701/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11388 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I808d55a225338912ebaebd0cf71c36db944b7276 Gerrit-Change-Number: 11388 Gerrit-PatchSet: 6 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Tue, 18 Sep 2018 16:40:40 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7348: fix race in KuduUtil.getKuduClient
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11423 ) Change subject: IMPALA-7348: fix race in KuduUtil.getKuduClient .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3171/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11423 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iec50e9b592593cea5418e7a074bdda065184086f Gerrit-Change-Number: 11423 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Tue, 18 Sep 2018 16:38:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7348: fix race in KuduUtil.getKuduClient
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11423 ) Change subject: IMPALA-7348: fix race in KuduUtil.getKuduClient .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11423 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iec50e9b592593cea5418e7a074bdda065184086f Gerrit-Change-Number: 11423 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Tue, 18 Sep 2018 16:38:02 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7348: fix race in KuduUtil.getKuduClient
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/11423 ) Change subject: IMPALA-7348: fix race in KuduUtil.getKuduClient .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11423 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iec50e9b592593cea5418e7a074bdda065184086f Gerrit-Change-Number: 11423 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Tue, 18 Sep 2018 16:36:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7579: use the generic filesystem client to delete and create files in test query profile contains all events.
Andrew Sherman has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11461 Change subject: IMPALA-7579: use the generic filesystem_client to delete and create files in test_query_profile_contains_all_events. .. IMPALA-7579: use the generic filesystem_client to delete and create files in test_query_profile_contains_all_events. This bug was introduced by IMPALA-6568 which added the test_query_profile_contains_all_events test. This test creates a file in the filesystem so that is can be used by 'load data inpath'. The test was using the hdfs_client object to do file system operations, but this client only works with hdfs. Switch to using the filesystem_client object which can work on other filesystems, including s3. Change-Id: I7c6e0899455dd4e12636b959fab4bde79f02fb95 --- M tests/query_test/test_observability.py 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/61/11461/1 -- To view, visit http://gerrit.cloudera.org:8080/11461 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I7c6e0899455dd4e12636b959fab4bde79f02fb95 Gerrit-Change-Number: 11461 Gerrit-PatchSet: 1 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Thomas Marshall
[Impala-ASF-CR] Fix caps and grammar of backend startup profile key
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11451 ) Change subject: Fix caps and grammar of backend startup profile key .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3169/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11451 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie7d38a81364a57de90a01f4f65a375dc2e403e18 Gerrit-Change-Number: 11451 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 18 Sep 2018 16:22:53 + Gerrit-HasComments: No
[Impala-ASF-CR] Fix caps and grammar of backend startup profile key
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11451 ) Change subject: Fix caps and grammar of backend startup profile key .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11451 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie7d38a81364a57de90a01f4f65a375dc2e403e18 Gerrit-Change-Number: 11451 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 18 Sep 2018 16:22:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7527: add fetch-from-catalogd cache info to profile
Vuk Ercegovac has uploaded a new patch set (#6) to the change originally created by Todd Lipcon. ( http://gerrit.cloudera.org:8080/11388 ) Change subject: IMPALA-7527: add fetch-from-catalogd cache info to profile .. IMPALA-7527: add fetch-from-catalogd cache info to profile This patch adds a Java wrapper for a RuntimeProfile object. The wrapper supports some basic operations like non-hierarchical counters and informational strings. During planning, a profile is created, and passed back to the backend as part of the ExecRequest. The backend then updates the query profile based on the info emitted from the frontend. This patch also adds the first use case for this profile information: the CatalogdMetaProvider emits counters for cache hits, misses, and fetch times, broken down by metadata category. The emitted profile is a bit of a superset of the existing 'timeline' functionality. However, it seems that some tools may parse the timeline in its current location in the profile, so moving it might be incompatible. I elected to leave that alone for now and just emit counters in the new profile. Change-Id: I808d55a225338912ebaebd0cf71c36db944b7276 --- M be/src/service/client-request-state.cc M be/src/service/client-request-state.h M be/src/service/impala-server.cc M be/src/util/runtime-profile.h M common/thrift/Frontend.thrift M fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java M fe/src/main/java/org/apache/impala/service/Frontend.java A fe/src/main/java/org/apache/impala/service/FrontendProfile.java M fe/src/test/java/org/apache/impala/catalog/local/CatalogdMetaProviderTest.java M tests/custom_cluster/test_local_catalog.py 10 files changed, 301 insertions(+), 15 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/11388/6 -- To view, visit http://gerrit.cloudera.org:8080/11388 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I808d55a225338912ebaebd0cf71c36db944b7276 Gerrit-Change-Number: 11388 Gerrit-PatchSet: 6 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] WIP IMPALA-7527: add fetch-from-catalogd cache info to profile
Vuk Ercegovac has posted comments on this change. (
http://gerrit.cloudera.org:8080/11388 )
Change subject: WIP IMPALA-7527: add fetch-from-catalogd cache info to profile
..
Patch Set 5:
(4 comments)
added e2e test, restricted nested scopes, added rpc metrics (per query, not
broken down by rpc method)
http://gerrit.cloudera.org:8080/#/c/11388/3/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java
File fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java:
http://gerrit.cloudera.org:8080/#/c/11388/3/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java@309
PS3, Line 309: throw new TException(e);
> Should we track num rpcs?
Done. These stats add the following lines (for a miss):
...
- CatalogFetch.RPCs.Requests: 4 (4)
- CatalogFetch.RPCs.Time: 337ms
- CatalogFetch.RPCs.Bytes: 33.28 KB (34082)
...
http://gerrit.cloudera.org:8080/#/c/11388/3/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java@349
PS3, Line 349: return (ValueType)cache_.get(key, new
Callable() {
> Does it help to implement a ScopedTimerWithStats (try-with-resources way)?
it might, but for three call-sites, I didn't think it was worth it right now.
http://gerrit.cloudera.org:8080/#/c/11388/3/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java@363
PS3, Line 363: temString, hit.getRef() ? "hit" : "miss");
: }
> this is kinda weird. Maybe overload this method that takes a single bool wh
its the only call-site with this 1 | 0 logic. the rest do not use it, so I
don't think its worth it to generalize it at this point.
http://gerrit.cloudera.org:8080/#/c/11388/3/fe/src/main/java/org/apache/impala/service/FrontendProfile.java
File fe/src/main/java/org/apache/impala/service/FrontendProfile.java:
http://gerrit.cloudera.org:8080/#/c/11388/3/fe/src/main/java/org/apache/impala/service/FrontendProfile.java@151
PS3, Line 151: oldThreadLocalValue_ = THREAD_LOCAL.get();
> k I think I'll just prevent nested scopes and document it as such
added the precondition and a todo to enable nested scopes.
--
To view, visit http://gerrit.cloudera.org:8080/11388
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I808d55a225338912ebaebd0cf71c36db944b7276
Gerrit-Change-Number: 11388
Gerrit-PatchSet: 5
Gerrit-Owner: Todd Lipcon
Gerrit-Reviewer: Andrew Sherman
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Todd Lipcon
Gerrit-Reviewer: Vuk Ercegovac
Gerrit-Comment-Date: Tue, 18 Sep 2018 16:05:56 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7525: [DOCS] SHOW GRANT USER
Adam Holley has posted comments on this change. ( http://gerrit.cloudera.org:8080/11454 ) Change subject: IMPALA-7525: [DOCS] SHOW GRANT USER .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/11454/1/docs/topics/impala_show.xml File docs/topics/impala_show.xml: http://gerrit.cloudera.org:8080/#/c/11454/1/docs/topics/impala_show.xml@516 PS1, Line 516: and other users that have been granted the : specified role. I'm not sure this is valid. What is the specified role? Additionally, the current user is allowed to run SHOW GRANT USER for themselves. -- To view, visit http://gerrit.cloudera.org:8080/11454 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iaa39275c595d73b2fba293f77bb21aa3985dfa38 Gerrit-Change-Number: 11454 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 18 Sep 2018 14:10:06 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Fix caps and grammar of backend startup profile key
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/11451 ) Change subject: Fix caps and grammar of backend startup profile key .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11451 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie7d38a81364a57de90a01f4f65a375dc2e403e18 Gerrit-Change-Number: 11451 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 18 Sep 2018 14:04:41 + Gerrit-HasComments: No
