[Impala-ASF-CR] Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog""
Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/17429 ) Change subject: Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog"" .. Patch Set 5: Couple of failures for the retry query feature. Since this patch ran cleanly in the original commit, I am guessing either the addendum causes some side effect or these are flaky tests? Perhaps re-submit to be sure. -- To view, visit http://gerrit.cloudera.org:8080/17429 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icbe93f3ae4efd585d4b0092a9ac7081b0b2c1c44 Gerrit-Change-Number: 17429 Gerrit-PatchSet: 5 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 13 May 2021 02:36:03 + Gerrit-HasComments: No
[native-toolchain-CR] IMPALA-10678: Bump Kudu version to f486f0813a
Wenzhe Zhou has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/17423 ) Change subject: IMPALA-10678: Bump Kudu version to f486f0813a .. IMPALA-10678: Bump Kudu version to f486f0813a This pulls in the latest version of Kudu which includes the patches for KUDU-1884 (Make Kerberos principal customizable). Testing: - Successfully built native-toolchain on Jenkins without publishing dependencies to S3. Change-Id: Ieafa9e5301297cd11032ea9cd5dbfa3fd4fa971a --- M buildall.sh 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/23/17423/2 -- To view, visit http://gerrit.cloudera.org:8080/17423 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ieafa9e5301297cd11032ea9cd5dbfa3fd4fa971a Gerrit-Change-Number: 17423 Gerrit-PatchSet: 2 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Joe McDonnell
[native-toolchain-CR] IMPALA-10678: Bump Kudu version to f486f0813a
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17423 ) Change subject: IMPALA-10678: Bump Kudu version to f486f0813a .. Patch Set 1: Code-Review+2 It makes sense to bump the Kudu version. For completeness, it would be best to do an Impala core run with the new artifacts. -- To view, visit http://gerrit.cloudera.org:8080/17423 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ieafa9e5301297cd11032ea9cd5dbfa3fd4fa971a Gerrit-Change-Number: 17423 Gerrit-PatchSet: 1 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Thu, 13 May 2021 00:58:37 + Gerrit-HasComments: No
[native-toolchain-CR] IMPALA-10488: Add jwt-cpp 0.5.0 to the toolchain
Joe McDonnell has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17118 ) Change subject: IMPALA-10488: Add jwt-cpp 0.5.0 to the toolchain .. IMPALA-10488: Add jwt-cpp 0.5.0 to the toolchain This adds a JSON Web Token (JWT) library for decoding and verifying JWTs. This requires OpenSSL 1.0.2. Tests: - Ran a build on all platforms Change-Id: I77aa3b36b45e8ef3c2d7873327948197c2c65d11 Reviewed-on: http://gerrit.cloudera.org:8080/17118 Reviewed-by: Wenzhe Zhou Reviewed-by: Joe McDonnell Tested-by: Joe McDonnell --- M buildall.sh A source/jwt-cpp/build.sh 2 files changed, 43 insertions(+), 0 deletions(-) Approvals: Wenzhe Zhou: Looks good to me, but someone else must approve Joe McDonnell: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/17118 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I77aa3b36b45e8ef3c2d7873327948197c2c65d11 Gerrit-Change-Number: 17118 Gerrit-PatchSet: 5 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Wenzhe Zhou
[native-toolchain-CR] IMPALA-10488: Add jwt-cpp 0.5.0 to the toolchain
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17118 ) Change subject: IMPALA-10488: Add jwt-cpp 0.5.0 to the toolchain .. Patch Set 4: Verified+1 Code-Review+2 Carrying +2. This passed a toolchain build, so merging. It doesn't interfere with any other changes to the toolchain. -- To view, visit http://gerrit.cloudera.org:8080/17118 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I77aa3b36b45e8ef3c2d7873327948197c2c65d11 Gerrit-Change-Number: 17118 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Thu, 13 May 2021 00:43:26 + Gerrit-HasComments: No
[Impala-ASF-CR] [WIP] IMPALA-10489: Implement JWT support
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17435 ) Change subject: [WIP] IMPALA-10489: Implement JWT support .. Patch Set 1: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/8712/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/17435 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6b71fa854c9ddc8ca882878853395e1eb866143c Gerrit-Change-Number: 17435 Gerrit-PatchSet: 1 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Thu, 13 May 2021 00:27:59 + Gerrit-HasComments: No
[Impala-ASF-CR] [WIP] IMPALA-10489: Implement JWT support
Wenzhe Zhou has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17435 Change subject: [WIP] IMPALA-10489: Implement JWT support .. [WIP] IMPALA-10489: Implement JWT support This patch added JWT support with following functionality: * Load and parse JWKS from pre-installed JSON file. * Read the JWT token from the HTTP Header. * Verify the JWT's signature with puclic key in JWKS. * Get the username out of the payload of JWT token. * Add unit-test for loading JWKS file and verifying JWT token. * TODO add more unit tests: FE custom cluster test for JWT authentication Testing: - TODO. Change-Id: I6b71fa854c9ddc8ca882878853395e1eb866143c --- M be/src/rpc/authentication.cc M be/src/service/impala-server.cc M be/src/service/impala-server.h M be/src/transport/THttpServer.cpp M be/src/transport/THttpServer.h M be/src/util/CMakeLists.txt A be/src/util/jwt-util-test.cc A be/src/util/jwt-util.cc A be/src/util/jwt-util.h M common/thrift/generate_error_codes.py M common/thrift/metrics.json 11 files changed, 927 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/35/17435/1 -- To view, visit http://gerrit.cloudera.org:8080/17435 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I6b71fa854c9ddc8ca882878853395e1eb866143c Gerrit-Change-Number: 17435 Gerrit-PatchSet: 1 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Joe McDonnell
[Impala-ASF-CR] Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog""
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17429 ) Change subject: Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog"" .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8711/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17429 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icbe93f3ae4efd585d4b0092a9ac7081b0b2c1c44 Gerrit-Change-Number: 17429 Gerrit-PatchSet: 5 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 12 May 2021 23:58:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10684: Report invalid parameter setting errors during HS2 OpenSession
Joe McDonnell has posted comments on this change. (
http://gerrit.cloudera.org:8080/17363 )
Change subject: IMPALA-10684: Report invalid parameter setting errors during
HS2 OpenSession
..
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/17363/2/be/src/service/impala-hs2-server.cc
File be/src/service/impala-hs2-server.cc:
http://gerrit.cloudera.org:8080/#/c/17363/2/be/src/service/impala-hs2-server.cc@364
PS2, Line 364: HS2_RETURN_IF_ERROR(return_val, SetQueryOption(v.first,
v.second,
: >set_query_options,
>set_query_options_mask),
: SQLSTATE_GENERAL_ERROR);
: if (iequals(v.first, "idle_session_timeout")) {
Another thing to consider is whether you want to detect all the bad query
options and merge the status into a single error message. We do that in
ParseQueryOptions here:
https://github.com/apache/impala/blob/master/be/src/service/query-options.cc#L1098-L1121
--
To view, visit http://gerrit.cloudera.org:8080/17363
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19a859bf5226feb903017a66f199ca5da916f217
Gerrit-Change-Number: 17363
Gerrit-PatchSet: 2
Gerrit-Owner: Kurt Deschler
Gerrit-Reviewer: Aman Sinha
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell
Gerrit-Reviewer: Kurt Deschler
Gerrit-Comment-Date: Wed, 12 May 2021 23:36:59 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog""
Vihang Karajgaonkar has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/17429 ) Change subject: Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog"" .. Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog"" This reverts commit 829d1a6ab4643b07877fb410971b67f1b1d1b045. Additionally, this also pulls in the addendum patch which was published here https://gerrit.cloudera.org/#/c/17330/ Change-Id: Icbe93f3ae4efd585d4b0092a9ac7081b0b2c1c44 --- M be/src/catalog/catalog-server.cc M be/src/common/global-flags.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M common/thrift/CatalogService.thrift M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java A fe/src/main/java/org/apache/impala/catalog/CatalogHmsAPIHelper.java M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java A fe/src/main/java/org/apache/impala/catalog/GetPartialCatalogObjectRequestBuilder.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/ParallelFileMetadataLoader.java M fe/src/main/java/org/apache/impala/catalog/Table.java A fe/src/main/java/org/apache/impala/catalog/metastore/CatalogHmsClientUtils.java A fe/src/main/java/org/apache/impala/catalog/metastore/CatalogMetastoreServer.java A fe/src/main/java/org/apache/impala/catalog/metastore/CatalogMetastoreServiceHandler.java A fe/src/main/java/org/apache/impala/catalog/metastore/ICatalogMetastoreServer.java A fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java A fe/src/main/java/org/apache/impala/catalog/metastore/NoOpCatalogMetastoreServer.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java A fe/src/test/java/org/apache/impala/catalog/metastore/CatalogHmsFileMetadataTest.java A fe/src/test/java/org/apache/impala/catalog/metastore/EnableCatalogdHmsCacheFlagTest.java M fe/src/test/java/org/apache/impala/testutil/CatalogServiceTestCatalog.java M tests/common/impala_test_suite.py A tests/custom_cluster/test_metastore_service.py 24 files changed, 5,398 insertions(+), 22 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/17429/5 -- To view, visit http://gerrit.cloudera.org:8080/17429 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icbe93f3ae4efd585d4b0092a9ac7081b0b2c1c44 Gerrit-Change-Number: 17429 Gerrit-PatchSet: 5 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-10680: Replace StringToFloatInternal using fast double parser library
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17389 ) Change subject: IMPALA-10680: Replace StringToFloatInternal using fast_double_parser library .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8710/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic105ad38a2fcbf2fb4e8ae8af6d9a8e251a9c141 Gerrit-Change-Number: 17389 Gerrit-PatchSet: 4 Gerrit-Owner: Amogh Margoor Gerrit-Reviewer: Amogh Margoor Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Wed, 12 May 2021 22:29:45 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10680: Replace StringToFloatInternal using fast double parser library
Amogh Margoor has posted comments on this change. ( http://gerrit.cloudera.org:8080/17389 ) Change subject: IMPALA-10680: Replace StringToFloatInternal using fast_double_parser library .. Patch Set 4: (2 comments) http://gerrit.cloudera.org:8080/#/c/17389/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/17389/3//COMMIT_MSG@10 PS3, Line 10: strto > strtod done. http://gerrit.cloudera.org:8080/#/c/17389/3//COMMIT_MSG@13 PS3, Line 13: accuracy > accuracy done. -- To view, visit http://gerrit.cloudera.org:8080/17389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic105ad38a2fcbf2fb4e8ae8af6d9a8e251a9c141 Gerrit-Change-Number: 17389 Gerrit-PatchSet: 4 Gerrit-Owner: Amogh Margoor Gerrit-Reviewer: Amogh Margoor Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Wed, 12 May 2021 22:08:41 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10680: Replace StringToFloatInternal using fast double parser library
Amogh Margoor has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/17389 ) Change subject: IMPALA-10680: Replace StringToFloatInternal using fast_double_parser library .. IMPALA-10680: Replace StringToFloatInternal using fast_double_parser library StringToFloatInternal is used to parse string into float. It had logic to ensure it is faster than standard functions like strtod in many cases, but it was not as accurate. We are replacing it by a third party library named fast_double_parser which is both fast and doesn't sacrifise the accuracy for speed. Testing: 1. Added test to check for accuracy improvement. 2. Ran existing Backend tests for correctness. Change-Id: Ic105ad38a2fcbf2fb4e8ae8af6d9a8e251a9c141 --- M be/src/exprs/expr-test.cc M be/src/util/string-parser-test.cc M be/src/util/string-parser.h M testdata/workloads/functional-query/queries/QueryTest/values.test 4 files changed, 72 insertions(+), 66 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/17389/4 -- To view, visit http://gerrit.cloudera.org:8080/17389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic105ad38a2fcbf2fb4e8ae8af6d9a8e251a9c141 Gerrit-Change-Number: 17389 Gerrit-PatchSet: 4 Gerrit-Owner: Amogh Margoor Gerrit-Reviewer: Amogh Margoor Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog""
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17429 ) Change subject: Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog"" .. Patch Set 3: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7134/ -- To view, visit http://gerrit.cloudera.org:8080/17429 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icbe93f3ae4efd585d4b0092a9ac7081b0b2c1c44 Gerrit-Change-Number: 17429 Gerrit-PatchSet: 3 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 12 May 2021 21:49:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10701: Switch to use TByteBuffer from thrift
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17428 ) Change subject: IMPALA-10701: Switch to use TByteBuffer from thrift .. Patch Set 3: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7133/ -- To view, visit http://gerrit.cloudera.org:8080/17428 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia0c7834253a16e440204264b0462a1590dea2463 Gerrit-Change-Number: 17428 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Wed, 12 May 2021 20:15:25 + Gerrit-HasComments: No
[native-toolchain-CR] IMPALA-10678: Bump Kudu version to f486f0813a
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17423 ) Change subject: IMPALA-10678: Bump Kudu version to f486f0813a .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/17423 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ieafa9e5301297cd11032ea9cd5dbfa3fd4fa971a Gerrit-Change-Number: 17423 Gerrit-PatchSet: 1 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Wed, 12 May 2021 20:11:26 + Gerrit-HasComments: No
[Impala-ASF-CR] Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog""
Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/17429 ) Change subject: Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog"" .. Patch Set 4: Code-Review+1 lgtm. There's one remaining formatting issue. -- To view, visit http://gerrit.cloudera.org:8080/17429 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icbe93f3ae4efd585d4b0092a9ac7081b0b2c1c44 Gerrit-Change-Number: 17429 Gerrit-PatchSet: 4 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 12 May 2021 19:35:44 + Gerrit-HasComments: No
[Impala-ASF-CR] Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog""
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17429 ) Change subject: Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog"" .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8709/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17429 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icbe93f3ae4efd585d4b0092a9ac7081b0b2c1c44 Gerrit-Change-Number: 17429 Gerrit-PatchSet: 4 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 12 May 2021 18:07:35 + Gerrit-HasComments: No
[Impala-ASF-CR] Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog""
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17429 ) Change subject: Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog"" .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/17429/4/common/thrift/BackendGflags.thrift File common/thrift/BackendGflags.thrift: http://gerrit.cloudera.org:8080/#/c/17429/4/common/thrift/BackendGflags.thrift@200 PS4, Line 200: line has trailing whitespace -- To view, visit http://gerrit.cloudera.org:8080/17429 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icbe93f3ae4efd585d4b0092a9ac7081b0b2c1c44 Gerrit-Change-Number: 17429 Gerrit-PatchSet: 4 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 12 May 2021 17:47:11 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog""
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/17429 ) Change subject: Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog"" .. Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/17429/2/common/thrift/BackendGflags.thrift File common/thrift/BackendGflags.thrift: http://gerrit.cloudera.org:8080/#/c/17429/2/common/thrift/BackendGflags.thrift@200 PS2, Line 200: > line has trailing whitespace Done http://gerrit.cloudera.org:8080/#/c/17429/2/tests/custom_cluster/test_metastore_service.py File tests/custom_cluster/test_metastore_service.py: http://gerrit.cloudera.org:8080/#/c/17429/2/tests/custom_cluster/test_metastore_service.py@215 PS2, Line 215: e > flake8: E722 do not use bare except' Done -- To view, visit http://gerrit.cloudera.org:8080/17429 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icbe93f3ae4efd585d4b0092a9ac7081b0b2c1c44 Gerrit-Change-Number: 17429 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 12 May 2021 17:46:47 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog""
Vihang Karajgaonkar has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/17429 ) Change subject: Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog"" .. Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog"" This reverts commit 829d1a6ab4643b07877fb410971b67f1b1d1b045. Additionally, this also pulls in the addendum patch which was published here https://gerrit.cloudera.org/#/c/17330/ Change-Id: Icbe93f3ae4efd585d4b0092a9ac7081b0b2c1c44 --- M be/src/catalog/catalog-server.cc M be/src/common/global-flags.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M common/thrift/CatalogService.thrift M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java A fe/src/main/java/org/apache/impala/catalog/CatalogHmsAPIHelper.java M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java A fe/src/main/java/org/apache/impala/catalog/GetPartialCatalogObjectRequestBuilder.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/ParallelFileMetadataLoader.java M fe/src/main/java/org/apache/impala/catalog/Table.java A fe/src/main/java/org/apache/impala/catalog/metastore/CatalogHmsClientUtils.java A fe/src/main/java/org/apache/impala/catalog/metastore/CatalogMetastoreServer.java A fe/src/main/java/org/apache/impala/catalog/metastore/CatalogMetastoreServiceHandler.java A fe/src/main/java/org/apache/impala/catalog/metastore/ICatalogMetastoreServer.java A fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java A fe/src/main/java/org/apache/impala/catalog/metastore/NoOpCatalogMetastoreServer.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java A fe/src/test/java/org/apache/impala/catalog/metastore/CatalogHmsFileMetadataTest.java A fe/src/test/java/org/apache/impala/catalog/metastore/EnableCatalogdHmsCacheFlagTest.java M fe/src/test/java/org/apache/impala/testutil/CatalogServiceTestCatalog.java M tests/common/impala_test_suite.py A tests/custom_cluster/test_metastore_service.py 24 files changed, 5,398 insertions(+), 22 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/17429/4 -- To view, visit http://gerrit.cloudera.org:8080/17429 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icbe93f3ae4efd585d4b0092a9ac7081b0b2c1c44 Gerrit-Change-Number: 17429 Gerrit-PatchSet: 4 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10613: (Addendum) Change isSetGetFileMetadata to isGetFileMetadata
Vihang Karajgaonkar has abandoned this change. ( http://gerrit.cloudera.org:8080/17330 ) Change subject: IMPALA-10613: (Addendum) Change isSetGetFileMetadata to isGetFileMetadata .. Abandoned -- To view, visit http://gerrit.cloudera.org:8080/17330 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: Ib96985e9865caaf5990cb9ef82787d076728eeff Gerrit-Change-Number: 17330 Gerrit-PatchSet: 1 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Anonymous Coward (646) Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-10613: (Addendum) Change isSetGetFileMetadata to isGetFileMetadata
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/17330 ) Change subject: IMPALA-10613: (Addendum) Change isSetGetFileMetadata to isGetFileMetadata .. Patch Set 1: Abandoning this since I plan to include this as part of https://gerrit.cloudera.org/#/c/17429/ -- To view, visit http://gerrit.cloudera.org:8080/17330 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib96985e9865caaf5990cb9ef82787d076728eeff Gerrit-Change-Number: 17330 Gerrit-PatchSet: 1 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Anonymous Coward (646) Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 12 May 2021 17:34:06 + Gerrit-HasComments: No
[Impala-ASF-CR] Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog""
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17429 ) Change subject: Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog"" .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8708/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17429 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icbe93f3ae4efd585d4b0092a9ac7081b0b2c1c44 Gerrit-Change-Number: 17429 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 12 May 2021 16:11:31 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10680: Replace StringToFloatInternal using fast double parser library
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/17389 ) Change subject: IMPALA-10680: Replace StringToFloatInternal using fast_double_parser library .. Patch Set 3: Code-Review+1 (2 comments) LGTM! http://gerrit.cloudera.org:8080/#/c/17389/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/17389/3//COMMIT_MSG@10 PS3, Line 10: strod strtod http://gerrit.cloudera.org:8080/#/c/17389/3//COMMIT_MSG@13 PS3, Line 13: accurancy accuracy -- To view, visit http://gerrit.cloudera.org:8080/17389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic105ad38a2fcbf2fb4e8ae8af6d9a8e251a9c141 Gerrit-Change-Number: 17389 Gerrit-PatchSet: 3 Gerrit-Owner: Amogh Margoor Gerrit-Reviewer: Amogh Margoor Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Wed, 12 May 2021 16:08:50 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog""
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17429 ) Change subject: Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog"" .. Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/17429/2/common/thrift/BackendGflags.thrift File common/thrift/BackendGflags.thrift: http://gerrit.cloudera.org:8080/#/c/17429/2/common/thrift/BackendGflags.thrift@200 PS2, Line 200: line has trailing whitespace http://gerrit.cloudera.org:8080/#/c/17429/2/tests/custom_cluster/test_metastore_service.py File tests/custom_cluster/test_metastore_service.py: http://gerrit.cloudera.org:8080/#/c/17429/2/tests/custom_cluster/test_metastore_service.py@215 PS2, Line 215: e flake8: E722 do not use bare except' -- To view, visit http://gerrit.cloudera.org:8080/17429 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icbe93f3ae4efd585d4b0092a9ac7081b0b2c1c44 Gerrit-Change-Number: 17429 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 12 May 2021 15:51:58 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog""
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17429 ) Change subject: Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog"" .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7134/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/17429 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icbe93f3ae4efd585d4b0092a9ac7081b0b2c1c44 Gerrit-Change-Number: 17429 Gerrit-PatchSet: 3 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 12 May 2021 15:51:41 + Gerrit-HasComments: No
[Impala-ASF-CR] Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog""
Vihang Karajgaonkar has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/17429 ) Change subject: Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog"" .. Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog"" This reverts commit 829d1a6ab4643b07877fb410971b67f1b1d1b045. Additionally, this also pulls in the addendum patch which was published here https://gerrit.cloudera.org/#/c/17330/ Change-Id: Icbe93f3ae4efd585d4b0092a9ac7081b0b2c1c44 --- M be/src/catalog/catalog-server.cc M be/src/common/global-flags.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M common/thrift/CatalogService.thrift M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java A fe/src/main/java/org/apache/impala/catalog/CatalogHmsAPIHelper.java M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java A fe/src/main/java/org/apache/impala/catalog/GetPartialCatalogObjectRequestBuilder.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/ParallelFileMetadataLoader.java M fe/src/main/java/org/apache/impala/catalog/Table.java A fe/src/main/java/org/apache/impala/catalog/metastore/CatalogHmsClientUtils.java A fe/src/main/java/org/apache/impala/catalog/metastore/CatalogMetastoreServer.java A fe/src/main/java/org/apache/impala/catalog/metastore/CatalogMetastoreServiceHandler.java A fe/src/main/java/org/apache/impala/catalog/metastore/ICatalogMetastoreServer.java A fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java A fe/src/main/java/org/apache/impala/catalog/metastore/NoOpCatalogMetastoreServer.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java A fe/src/test/java/org/apache/impala/catalog/metastore/CatalogHmsFileMetadataTest.java A fe/src/test/java/org/apache/impala/catalog/metastore/EnableCatalogdHmsCacheFlagTest.java M fe/src/test/java/org/apache/impala/testutil/CatalogServiceTestCatalog.java M tests/common/impala_test_suite.py A tests/custom_cluster/test_metastore_service.py 24 files changed, 5,398 insertions(+), 22 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/17429/2 -- To view, visit http://gerrit.cloudera.org:8080/17429 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icbe93f3ae4efd585d4b0092a9ac7081b0b2c1c44 Gerrit-Change-Number: 17429 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog""
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17429 ) Change subject: Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog"" .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8707/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17429 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icbe93f3ae4efd585d4b0092a9ac7081b0b2c1c44 Gerrit-Change-Number: 17429 Gerrit-PatchSet: 1 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 12 May 2021 14:51:26 + Gerrit-HasComments: No
[Impala-ASF-CR] Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog""
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17429 ) Change subject: Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog"" .. Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/17429/1/common/thrift/BackendGflags.thrift File common/thrift/BackendGflags.thrift: http://gerrit.cloudera.org:8080/#/c/17429/1/common/thrift/BackendGflags.thrift@200 PS1, Line 200: line has trailing whitespace http://gerrit.cloudera.org:8080/#/c/17429/1/tests/custom_cluster/test_metastore_service.py File tests/custom_cluster/test_metastore_service.py: http://gerrit.cloudera.org:8080/#/c/17429/1/tests/custom_cluster/test_metastore_service.py@215 PS1, Line 215: e flake8: E722 do not use bare except' -- To view, visit http://gerrit.cloudera.org:8080/17429 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icbe93f3ae4efd585d4b0092a9ac7081b0b2c1c44 Gerrit-Change-Number: 17429 Gerrit-PatchSet: 1 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 12 May 2021 14:32:31 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog""
Vihang Karajgaonkar has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17429 Change subject: Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog"" .. Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog"" This reverts commit 829d1a6ab4643b07877fb410971b67f1b1d1b045. Change-Id: Icbe93f3ae4efd585d4b0092a9ac7081b0b2c1c44 --- M be/src/catalog/catalog-server.cc M be/src/common/global-flags.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M common/thrift/CatalogService.thrift M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java A fe/src/main/java/org/apache/impala/catalog/CatalogHmsAPIHelper.java M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java A fe/src/main/java/org/apache/impala/catalog/GetPartialCatalogObjectRequestBuilder.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/ParallelFileMetadataLoader.java M fe/src/main/java/org/apache/impala/catalog/Table.java A fe/src/main/java/org/apache/impala/catalog/metastore/CatalogHmsClientUtils.java A fe/src/main/java/org/apache/impala/catalog/metastore/CatalogMetastoreServer.java A fe/src/main/java/org/apache/impala/catalog/metastore/CatalogMetastoreServiceHandler.java A fe/src/main/java/org/apache/impala/catalog/metastore/ICatalogMetastoreServer.java A fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java A fe/src/main/java/org/apache/impala/catalog/metastore/NoOpCatalogMetastoreServer.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java A fe/src/test/java/org/apache/impala/catalog/metastore/CatalogHmsFileMetadataTest.java A fe/src/test/java/org/apache/impala/catalog/metastore/EnableCatalogdHmsCacheFlagTest.java M fe/src/test/java/org/apache/impala/testutil/CatalogServiceTestCatalog.java M tests/common/impala_test_suite.py A tests/custom_cluster/test_metastore_service.py 24 files changed, 5,398 insertions(+), 22 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/17429/1 -- To view, visit http://gerrit.cloudera.org:8080/17429 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Icbe93f3ae4efd585d4b0092a9ac7081b0b2c1c44 Gerrit-Change-Number: 17429 Gerrit-PatchSet: 1 Gerrit-Owner: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-10687: Implement ds cpc union() function
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17372 ) Change subject: IMPALA-10687: Implement ds_cpc_union() function .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8706/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17372 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib94b45ae79efcc11adc077dd9df9b9868ae82cb6 Gerrit-Change-Number: 17372 Gerrit-PatchSet: 4 Gerrit-Owner: Fucun Chu Gerrit-Reviewer: Fucun Chu Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 12 May 2021 14:29:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10675: Exclude hive-exec from ranger-plugins-audit
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/17339 ) Change subject: IMPALA-10675: Exclude hive-exec from ranger-plugins-audit .. Patch Set 2: I think this can be abandoned now since I just merged https://gerrit.cloudera.org/#/c/17424/ which includes this change. -- To view, visit http://gerrit.cloudera.org:8080/17339 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I459360028e12dabbdbaff2250f1b712b6f165f5c Gerrit-Change-Number: 17339 Gerrit-PatchSet: 2 Gerrit-Owner: Fang-Yu Rao Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 12 May 2021 14:29:01 + Gerrit-HasComments: No
[Impala-ASF-CR] Bump up GBN to 13088653
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/17424 ) Change subject: Bump up GBN to 13088653 .. Patch Set 1: > Aman just gave a +2 for my patch at https://gerrit.cloudera.org/c/17339/. > So I think this patch could be rebased against the master branch > after IMPALA-10675 (Exclude hive-exec from ranger-plugins-audit) is > merged. :-) It looks like https://gerrit.cloudera.org/c/17339/ could not be merged because the GVO failed during dataload. Since this patch already passed the GVO job, I think it is easier to merge this one now and abandon https://gerrit.cloudera.org/c/17339/ -- To view, visit http://gerrit.cloudera.org:8080/17424 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie3ad6311f0ef1a06391b70d58cdb4e51e08de5ea Gerrit-Change-Number: 17424 Gerrit-PatchSet: 1 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 12 May 2021 14:28:06 + Gerrit-HasComments: No
[Impala-ASF-CR] Bump up GBN to 13088653
Vihang Karajgaonkar has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17424 ) Change subject: Bump up GBN to 13088653 .. Bump up GBN to 13088653 This change bumps up the CDP_BUILD_NUMBER to 13088653 which includes additional metastore thrift API changes needed for Catalogd's metastore endpoint to be functional. Additionally, this adds a exclusion in the ranger-audit-plugin dependency because it leaks hive-exec jar in the classpath which causes thrift errors during compilation. The compilation error occur because hive-exec leaks 0.9.3 thrift while impala now uses 0.11.0 version of thrift. Testing: 1. Verified that code builds locally. 2. Run core tests. Change-Id: Ie3ad6311f0ef1a06391b70d58cdb4e51e08de5ea Reviewed-on: http://gerrit.cloudera.org:8080/17424 Reviewed-by: Fang-Yu Rao Tested-by: Impala Public Jenkins Reviewed-by: Csaba Ringhofer --- M bin/impala-config.sh M fe/pom.xml 2 files changed, 20 insertions(+), 12 deletions(-) Approvals: Fang-Yu Rao: Looks good to me, but someone else must approve Impala Public Jenkins: Verified Csaba Ringhofer: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/17424 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ie3ad6311f0ef1a06391b70d58cdb4e51e08de5ea Gerrit-Change-Number: 17424 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-10687: Implement ds cpc union() function
Fucun Chu has posted comments on this change. ( http://gerrit.cloudera.org:8080/17372 ) Change subject: IMPALA-10687: Implement ds_cpc_union() function .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/17372/1/testdata/workloads/functional-query/queries/QueryTest/datasketches-cpc.test File testdata/workloads/functional-query/queries/QueryTest/datasketches-cpc.test: http://gerrit.cloudera.org:8080/#/c/17372/1/testdata/workloads/functional-query/queries/QueryTest/datasketches-cpc.test@192 PS1, Line 192: # result as if the whole data was sketched together into a single sketch. > I checked the test above that are run on functional_parquet.alltypessmall, Done -- To view, visit http://gerrit.cloudera.org:8080/17372 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib94b45ae79efcc11adc077dd9df9b9868ae82cb6 Gerrit-Change-Number: 17372 Gerrit-PatchSet: 4 Gerrit-Owner: Fucun Chu Gerrit-Reviewer: Fucun Chu Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 12 May 2021 14:25:33 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10701: Switch to use TByteBuffer from thrift
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17428 ) Change subject: IMPALA-10701: Switch to use TByteBuffer from thrift .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17428 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia0c7834253a16e440204264b0462a1590dea2463 Gerrit-Change-Number: 17428 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Wed, 12 May 2021 14:18:36 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10701: Switch to use TByteBuffer from thrift
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17428 ) Change subject: IMPALA-10701: Switch to use TByteBuffer from thrift .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7133/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/17428 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia0c7834253a16e440204264b0462a1590dea2463 Gerrit-Change-Number: 17428 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Wed, 12 May 2021 14:18:37 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10687: Implement ds cpc union() function
Hello Gabor Kaszab, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17372 to look at the new patch set (#4). Change subject: IMPALA-10687: Implement ds_cpc_union() function .. IMPALA-10687: Implement ds_cpc_union() function This function receives a set of serialized Apache DataSketches CPC sketches produced by ds_cpc_sketch() and merges them into a single sketch. An example usage is to create a sketch for each partition of a table, write these sketches to a separate table and based on which partition the user is interested of the relevant sketches can be union-ed together to get an estimate. E.g.: SELECT ds_cpc_estimate(ds_cpc_union(sketch_col)) FROM sketch_tbl WHERE partition_col=1 OR partition_col=5; Testing: - Apart from the automated tests I added to this patch I also tested ds_cpc_union() on a bigger dataset to check that serialization, deserialization and merging steps work well. I took TPCH25.linelitem, created a number of sketches with grouping by l_shipdate and called ds_cpc_union() on those sketches Change-Id: Ib94b45ae79efcc11adc077dd9df9b9868ae82cb6 --- M be/src/exprs/aggregate-functions-ir.cc M be/src/exprs/aggregate-functions.h M fe/src/main/java/org/apache/impala/catalog/BuiltinsDb.java M testdata/data/README A testdata/data/cpc_sketches_from_impala.parquet M testdata/workloads/functional-query/queries/QueryTest/datasketches-cpc.test M tests/query_test/test_datasketches.py 7 files changed, 177 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/72/17372/4 -- To view, visit http://gerrit.cloudera.org:8080/17372 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib94b45ae79efcc11adc077dd9df9b9868ae82cb6 Gerrit-Change-Number: 17372 Gerrit-PatchSet: 4 Gerrit-Owner: Fucun Chu Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10687: Implement ds cpc union() function
Gabor Kaszab has posted comments on this change. ( http://gerrit.cloudera.org:8080/17372 ) Change subject: IMPALA-10687: Implement ds_cpc_union() function .. Patch Set 3: Code-Review+1 (1 comment) http://gerrit.cloudera.org:8080/#/c/17372/1/testdata/workloads/functional-query/queries/QueryTest/datasketches-cpc.test File testdata/workloads/functional-query/queries/QueryTest/datasketches-cpc.test: http://gerrit.cloudera.org:8080/#/c/17372/1/testdata/workloads/functional-query/queries/QueryTest/datasketches-cpc.test@192 PS1, Line 192: ds_cpc_estimate(ds_cpc_union(date_sketch)), I checked the test above that are run on functional_parquet.alltypessmall, and they don't include date_string_col (that serves as the source for 'date_sketch'). Could you add that column to those tests so that we can compare the value '12' here to the one there? -- To view, visit http://gerrit.cloudera.org:8080/17372 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib94b45ae79efcc11adc077dd9df9b9868ae82cb6 Gerrit-Change-Number: 17372 Gerrit-PatchSet: 3 Gerrit-Owner: Fucun Chu Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 12 May 2021 13:05:10 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Bump up GBN to 13088653
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/17424 ) Change subject: Bump up GBN to 13088653 .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17424 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie3ad6311f0ef1a06391b70d58cdb4e51e08de5ea Gerrit-Change-Number: 17424 Gerrit-PatchSet: 1 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 12 May 2021 12:03:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10687: Implement ds cpc union() function
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17372 ) Change subject: IMPALA-10687: Implement ds_cpc_union() function .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8705/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17372 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib94b45ae79efcc11adc077dd9df9b9868ae82cb6 Gerrit-Change-Number: 17372 Gerrit-PatchSet: 3 Gerrit-Owner: Fucun Chu Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 12 May 2021 11:32:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10701: Switch to use TByteBuffer from thrift
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/17428 ) Change subject: IMPALA-10701: Switch to use TByteBuffer from thrift .. Patch Set 2: Code-Review+2 Thanks for cleaning this up! -- To view, visit http://gerrit.cloudera.org:8080/17428 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia0c7834253a16e440204264b0462a1590dea2463 Gerrit-Change-Number: 17428 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Wed, 12 May 2021 11:23:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10687: Implement ds cpc union() function
Fucun Chu has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/17372 ) Change subject: IMPALA-10687: Implement ds_cpc_union() function .. IMPALA-10687: Implement ds_cpc_union() function This function receives a set of serialized Apache DataSketches CPC sketches produced by ds_cpc_sketch() and merges them into a single sketch. An example usage is to create a sketch for each partition of a table, write these sketches to a separate table and based on which partition the user is interested of the relevant sketches can be union-ed together to get an estimate. E.g.: SELECT ds_cpc_estimate(ds_cpc_union(sketch_col)) FROM sketch_tbl WHERE partition_col=1 OR partition_col=5; Testing: - Apart from the automated tests I added to this patch I also tested ds_cpc_union() on a bigger dataset to check that serialization, deserialization and merging steps work well. I took TPCH25.linelitem, created a number of sketches with grouping by l_shipdate and called ds_cpc_union() on those sketches Change-Id: Ib94b45ae79efcc11adc077dd9df9b9868ae82cb6 --- M be/src/exprs/aggregate-functions-ir.cc M be/src/exprs/aggregate-functions.h M fe/src/main/java/org/apache/impala/catalog/BuiltinsDb.java M testdata/data/README A testdata/data/cpc_sketches_from_impala.parquet M testdata/workloads/functional-query/queries/QueryTest/datasketches-cpc.test M tests/query_test/test_datasketches.py 7 files changed, 169 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/72/17372/3 -- To view, visit http://gerrit.cloudera.org:8080/17372 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib94b45ae79efcc11adc077dd9df9b9868ae82cb6 Gerrit-Change-Number: 17372 Gerrit-PatchSet: 3 Gerrit-Owner: Fucun Chu Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10701: Switch to use TByteBuffer from thrift
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17428 ) Change subject: IMPALA-10701: Switch to use TByteBuffer from thrift .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8704/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17428 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia0c7834253a16e440204264b0462a1590dea2463 Gerrit-Change-Number: 17428 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Wed, 12 May 2021 10:35:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7556: Decouple BufferManagement from the ScanRange and IoMgr
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17413 ) Change subject: IMPALA-7556: Decouple BufferManagement from the ScanRange and IoMgr .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8703/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17413 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibd74691b50b46114f95a8641034c05d07ddeec97 Gerrit-Change-Number: 17413 Gerrit-PatchSet: 4 Gerrit-Owner: Amogh Margoor Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Wed, 12 May 2021 10:25:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10701: Switch to use TByteBuffer from thrift
Hello Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17428 to look at the new patch set (#2). Change subject: IMPALA-10701: Switch to use TByteBuffer from thrift .. IMPALA-10701: Switch to use TByteBuffer from thrift We have a copy of the TByteBuffer when we are using thrift versions prior to 0.10. After IMPALA-7825 bumps the thrift version to 0.11.0, we can use it directly. Tests - Run CORE tests Change-Id: Ia0c7834253a16e440204264b0462a1590dea2463 --- M fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java M fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java D fe/src/main/java/org/apache/impala/util/TByteBuffer.java 3 files changed, 2 insertions(+), 64 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/28/17428/2 -- To view, visit http://gerrit.cloudera.org:8080/17428 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia0c7834253a16e440204264b0462a1590dea2463 Gerrit-Change-Number: 17428 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-10701: Switch to use TByteBuffer from thrift
Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17428 Change subject: IMPALA-10701: Switch to use TByteBuffer from thrift .. IMPALA-10701: Switch to use TByteBuffer from thrift We have a copy of the TByteBuffer when we are using thrift versions prior to 0.10. After IMPALA-7825 the thrift version to 0.11.0, we can use it directly. Tests - Run CORE tests Change-Id: Ia0c7834253a16e440204264b0462a1590dea2463 --- M fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java M fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java D fe/src/main/java/org/apache/impala/util/TByteBuffer.java 3 files changed, 2 insertions(+), 64 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/28/17428/1 -- To view, visit http://gerrit.cloudera.org:8080/17428 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ia0c7834253a16e440204264b0462a1590dea2463 Gerrit-Change-Number: 17428 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang
[Impala-ASF-CR] IMPALA-7556: Decouple BufferManagement from the ScanRange and IoMgr
Amogh Margoor has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/17413 ) Change subject: IMPALA-7556: Decouple BufferManagement from the ScanRange and IoMgr .. IMPALA-7556: Decouple BufferManagement from the ScanRange and IoMgr Currently BufferManagement is tightly coupled with ScanRange. Every ScanRange maintains list of unused buffers and ready buffers. Unused buffers are buffers used to read scanned data and ready buffers are buffers with the data already read. For managing these buffers, ScanRange defines various functions like AddUnusedBuffer, GetUsedBuffer, EnqueueReadyBuffer and functions to allocate and cleanup buffers. This patch has created ScanBufferManager which would be responsible for the managing these buffers for ScanRange. ScanBufferManager's logic is still coupled with the ScanRange, but refactorig it into a seperate class is a good first step. As ScanRange's lock is used to synchronize various functions including the buffer management, a new class for lock store has been created too. Testing: 1. Ran these existing tests: EE, BackEnd, JDBC and Cluster test. Change-Id: Ibd74691b50b46114f95a8641034c05d07ddeec97 --- M be/src/runtime/io/CMakeLists.txt M be/src/runtime/io/disk-io-mgr.cc M be/src/runtime/io/request-context.cc M be/src/runtime/io/request-context.h M be/src/runtime/io/request-ranges.h A be/src/runtime/io/scan-buffer-manager.cc A be/src/runtime/io/scan-buffer-manager.h M be/src/runtime/io/scan-range.cc 8 files changed, 531 insertions(+), 282 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/13/17413/4 -- To view, visit http://gerrit.cloudera.org:8080/17413 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ibd74691b50b46114f95a8641034c05d07ddeec97 Gerrit-Change-Number: 17413 Gerrit-PatchSet: 4 Gerrit-Owner: Amogh Margoor Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-10702: Add warning logs for slow or large catalogd response
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17427 ) Change subject: IMPALA-10702: Add warning logs for slow or large catalogd response .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8702/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17427 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icffcfcaad2a718aebf79e2331efb05ca7a9a7671 Gerrit-Change-Number: 17427 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 12 May 2021 08:53:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10702: Add warning logs for slow or large catalogd response
Impala Public Jenkins has posted comments on this change. (
http://gerrit.cloudera.org:8080/17427 )
Change subject: IMPALA-10702: Add warning logs for slow or large catalogd
response
..
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/17427/1/fe/src/main/java/org/apache/impala/service/BackendConfig.java
File fe/src/main/java/org/apache/impala/service/BackendConfig.java:
http://gerrit.cloudera.org:8080/#/c/17427/1/fe/src/main/java/org/apache/impala/service/BackendConfig.java@273
PS1, Line 273: public long getWarnCatalogResponseDurationMs() { return
warnCatalogResponseDurationMs_; }
line too long (91 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/17427
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Icffcfcaad2a718aebf79e2331efb05ca7a9a7671
Gerrit-Change-Number: 17427
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Wed, 12 May 2021 08:33:15 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10702: Add warning logs for slow or large catalogd response
Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17427 Change subject: IMPALA-10702: Add warning logs for slow or large catalogd response .. IMPALA-10702: Add warning logs for slow or large catalogd response It'd be helpful to log the slow or large responses of catalogd in debugging scalability issues. This patch adds these warning logs in JniCatalog, where we serialize thrift responses. See some example outputs in the jira description. Responses that have size larger than 50MB or take more than 60s to finish will be logged with the request. Add flags for these two thredshold in case users found the warnings too verbose and want to increase the thresholds. Change-Id: Icffcfcaad2a718aebf79e2331efb05ca7a9a7671 --- M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/main/java/org/apache/impala/common/JniUtil.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/main/java/org/apache/impala/service/JniCatalog.java 5 files changed, 127 insertions(+), 23 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/27/17427/1 -- To view, visit http://gerrit.cloudera.org:8080/17427 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Icffcfcaad2a718aebf79e2331efb05ca7a9a7671 Gerrit-Change-Number: 17427 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang
[Impala-ASF-CR] IMPALA-10401: Enable Ranger audit logs to log4j
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17416 ) Change subject: IMPALA-10401: Enable Ranger audit logs to log4j .. IMPALA-10401: Enable Ranger audit logs to log4j This patch enables audit logging to log4j for Ranger plugins, which provides us with more info in debugging flaky test failures. The audits are logged by Log4JAuditDestination. Each line shows which policy takes effect and whether the access is allowed. For FE tests, the log file is in logs/fe_tests/FeSupport.INFO. For e2e and custom cluster tests, the audits are logged in log files of processes that have ranger plugins (e.g. catalogd, cooridnator). Tests: - Manually verify the audit logs exist. Change-Id: I9e6850882594f31dc2250fe205fa6cad26a30571 Reviewed-on: http://gerrit.cloudera.org:8080/17416 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M fe/src/test/resources/ranger-hive-audit.xml 1 file changed, 5 insertions(+), 1 deletion(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/17416 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I9e6850882594f31dc2250fe205fa6cad26a30571 Gerrit-Change-Number: 17416 Gerrit-PatchSet: 4 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-10401: Enable Ranger audit logs to log4j
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17416 ) Change subject: IMPALA-10401: Enable Ranger audit logs to log4j .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/17416 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9e6850882594f31dc2250fe205fa6cad26a30571 Gerrit-Change-Number: 17416 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Wed, 12 May 2021 07:05:50 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10413: fix impalad crashes when canceling the retrying query
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16911 ) Change subject: IMPALA-10413: fix impalad crashes when canceling the retrying query .. IMPALA-10413: fix impalad crashes when canceling the retrying query The crash happens when canceling the retrying query. If the original query was unregistered while the new query was being created, it will call HandleRetryFailure to abort the new query. But the status is ok, so when calling Status::AddDetail impalad will crash. After the WaitAsync interface called and before the retry_request_state moved to retried_client_request_state_ , if abort the new retry query, retry_request_state need to call Finalize, otherwise the wait-thread will leak. In some cases like canceled the original query or closed the session we may not create the new query, so we also check whether the query is retried. Tests: Add test in tests/custom_cluster/test_query_retries.py and manually tested 100 times to make sure that there was no Impalad crash Change-Id: I4fd7228acd0a70d33859029052239f9b9f795e5d Reviewed-on: http://gerrit.cloudera.org:8080/16911 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/runtime/query-driver.cc M be/src/service/client-request-state.h M be/src/service/impala-server.cc M tests/custom_cluster/test_query_retries.py 4 files changed, 106 insertions(+), 14 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/16911 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I4fd7228acd0a70d33859029052239f9b9f795e5d Gerrit-Change-Number: 16911 Gerrit-PatchSet: 16 Gerrit-Owner: Xianqing He Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Wenzhe Zhou Gerrit-Reviewer: Xianqing He
[Impala-ASF-CR] IMPALA-10413: fix impalad crashes when canceling the retrying query
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16911 ) Change subject: IMPALA-10413: fix impalad crashes when canceling the retrying query .. Patch Set 15: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/16911 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4fd7228acd0a70d33859029052239f9b9f795e5d Gerrit-Change-Number: 16911 Gerrit-PatchSet: 15 Gerrit-Owner: Xianqing He Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Wenzhe Zhou Gerrit-Reviewer: Xianqing He Gerrit-Comment-Date: Wed, 12 May 2021 06:37:42 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10675: Exclude hive-exec from ranger-plugins-audit
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17339 ) Change subject: IMPALA-10675: Exclude hive-exec from ranger-plugins-audit .. Patch Set 2: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7131/ -- To view, visit http://gerrit.cloudera.org:8080/17339 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I459360028e12dabbdbaff2250f1b712b6f165f5c Gerrit-Change-Number: 17339 Gerrit-PatchSet: 2 Gerrit-Owner: Fang-Yu Rao Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Wed, 12 May 2021 06:28:31 + Gerrit-HasComments: No
