[Impala-ASF-CR] IMPALA-8778: Support Apache Hudi Read Optimized Table
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/14711 ) Change subject: IMPALA-8778: Support Apache Hudi Read Optimized Table .. Patch Set 22: (1 comment) http://gerrit.cloudera.org:8080/#/c/14711/22/testdata/data/README File testdata/data/README: http://gerrit.cloudera.org:8080/#/c/14711/22/testdata/data/README@489 PS22, Line 489: `ca51fa17-681b-4497-85b7-4f68e7a63ee7-0_1-5-10_20200112194517.parquet` : `ca51fa17-681b-4497-85b7-4f68e7a63ee7-0` is the bloom index hash of this file : `20200112194517` is the timestamp of this version > Thanks for pointing this out. 10KB seems fine to me, there are lot of files in the repo around that size. -- To view, visit http://gerrit.cloudera.org:8080/14711 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I65e146b347714df32fe968409ef2dde1f6a25cdf Gerrit-Change-Number: 14711 Gerrit-PatchSet: 22 Gerrit-Owner: Yanjia Gary Li Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Yanjia Gary Li Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 10 Feb 2020 12:34:23 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 ) Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/15189 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42 Gerrit-Change-Number: 15189 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Mon, 10 Feb 2020 12:54:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 ) Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/15189/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/15189/1//COMMIT_MSG@20 PS1, Line 20: Tests Did you think about adding some kind of custom cluster test to prevent bit rot? It would be good to check that updating Ranger/Hive dependencies do not break this. It could start as a very simple Hive query that needs Ranger, but it may worth to extend it in the future if we need Ranger related interop tests. -- To view, visit http://gerrit.cloudera.org:8080/15189 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42 Gerrit-Change-Number: 15189 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Mon, 10 Feb 2020 12:54:23 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9242: Filter privileges before returning them to Sentry
Hello Anonymous Coward (498), Vihang Karajgaonkar, Daniel Becker, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/15068 to look at the new patch set (#9). Change subject: IMPALA-9242: Filter privileges before returning them to Sentry .. IMPALA-9242: Filter privileges before returning them to Sentry This change implements the new FilteredPrivilegeCache, which adds functions for filtering privileges based on the authorizable and for returning Privileges directly instead of their String form. The filtering is based on server + db + table (or just server in case of URI privileges) to filter out the bulk of unrelated privileges. Efficient filtering is done by a new class PrincipalPrivilegeTree. It was tempting to reuse Sentry's TreePrivilegeCache, which has a very similar role, but it lacks a "remove" function that is needed to keep this index in sync with the CatalogObjectCache in Principal. I am also a bit concerned about the possible side effect of Sentry's interning of names in privileges - we try to avoid using String.intern() on massive amount of names in Impala. Other Changes: - Add the Sentry privilege name as member to PrincipalPrivileges. Note that the name was a member of TPrivilege till IMPALA-7616. Storing the name shouldn't consume much extra memory, as it is already stored as the key of the PrincipalPrivilege in CatalogObjectCache. Testing: - added unit tests based on Sentry / TestTreePrivilegeCache Change-Id: Iecd4281368d1c9fe88cfe850ea725cd68895712e --- M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationPolicy.java M fe/src/main/java/org/apache/impala/catalog/Principal.java M fe/src/main/java/org/apache/impala/catalog/PrincipalPrivilege.java A fe/src/main/java/org/apache/impala/catalog/PrincipalPrivilegeTree.java A fe/src/test/java/org/apache/impala/catalog/PrincipalPrivilegeTreeTest.java 5 files changed, 486 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/15068/9 -- To view, visit http://gerrit.cloudera.org:8080/15068 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iecd4281368d1c9fe88cfe850ea725cd68895712e Gerrit-Change-Number: 15068 Gerrit-PatchSet: 9 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Anonymous Coward (498) Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-9242: Filter privileges before returning them to Sentry
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15068 ) Change subject: IMPALA-9242: Filter privileges before returning them to Sentry .. Patch Set 9: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5181/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15068 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iecd4281368d1c9fe88cfe850ea725cd68895712e Gerrit-Change-Number: 15068 Gerrit-PatchSet: 9 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Anonymous Coward (498) Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Mon, 10 Feb 2020 14:59:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9242: Filter privileges before returning them to Sentry
Hello Anonymous Coward (498), Vihang Karajgaonkar, Daniel Becker, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/15068 to look at the new patch set (#10). Change subject: IMPALA-9242: Filter privileges before returning them to Sentry .. IMPALA-9242: Filter privileges before returning them to Sentry This change implements the new FilteredPrivilegeCache, which adds functions for filtering privileges based on the authorizable and for returning Privileges directly instead of their String form. The filtering is based on server + db + table (or just server in case of URI privileges) to filter out the bulk of unrelated privileges. Efficient filtering is done by a new class PrincipalPrivilegeTree. It was tempting to reuse Sentry's TreePrivilegeCache, which has a very similar role, but it lacks a "remove" function that is needed to keep this index in sync with the CatalogObjectCache in Principal. I am also a bit concerned about the possible side effect of Sentry's interning of names in privileges - we try to avoid using String.intern() on massive amount of names in Impala. Other Changes: - Add the Sentry privilege name as member to PrincipalPrivileges. Note that the name was a member of TPrivilege till IMPALA-7616. Storing the name shouldn't consume much extra memory, as it is already stored as the key of the PrincipalPrivilege in CatalogObjectCache. Testing: - added unit tests based on Sentry / TestTreePrivilegeCache Change-Id: Iecd4281368d1c9fe88cfe850ea725cd68895712e --- M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationPolicy.java M fe/src/main/java/org/apache/impala/catalog/Principal.java M fe/src/main/java/org/apache/impala/catalog/PrincipalPrivilege.java A fe/src/main/java/org/apache/impala/catalog/PrincipalPrivilegeTree.java A fe/src/test/java/org/apache/impala/catalog/PrincipalPrivilegeTreeTest.java 5 files changed, 506 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/15068/10 -- To view, visit http://gerrit.cloudera.org:8080/15068 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iecd4281368d1c9fe88cfe850ea725cd68895712e Gerrit-Change-Number: 15068 Gerrit-PatchSet: 10 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Anonymous Coward (498) Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[native-toolchain-CR] IMPALA-9279: part 2: Bump Kudu version to 5c610bf40
Attila Jeges has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15192 Change subject: IMPALA-9279: part 2: Bump Kudu version to 5c610bf40 .. IMPALA-9279: part 2: Bump Kudu version to 5c610bf40 This pulls in a Kudu change that links Kudu executables statically to libcurl in Kudu's thirdparty directory instead of relying on the dynamic linker to find libcurl at runtime. Testing: - Ran the C6 toolchain build job with the Kudu version bump for native toolchain to make sure that it builds on all supported platforms. Change-Id: I3ff92cc5e1de220a4c140cf6c0117b5fa1e89226 --- M buildall.sh 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/92/15192/1 -- To view, visit http://gerrit.cloudera.org:8080/15192 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I3ff92cc5e1de220a4c140cf6c0117b5fa1e89226 Gerrit-Change-Number: 15192 Gerrit-PatchSet: 1 Gerrit-Owner: Attila Jeges
[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support
Attila Jeges has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 ) Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support .. Patch Set 6: > I just saw this commit in Kudu, and I'm wondering if it helps with > the libcurl situation: https://gerrit.cloudera.org/#/c/15180/ > > If they link libcurl, then we might not need to install it. Correct, this Kudu change eliminates the runtime dependency on the libcurl shared library. Here's a native-toolchain CR to bump kudu version once again to include the fix: https://gerrit.cloudera.org/#/c/15192/ -- To view, visit http://gerrit.cloudera.org:8080/15134 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a Gerrit-Change-Number: 15134 Gerrit-PatchSet: 6 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Mon, 10 Feb 2020 15:43:46 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9242: Filter privileges before returning them to Sentry
Csaba Ringhofer has posted comments on this change. (
http://gerrit.cloudera.org:8080/15068 )
Change subject: IMPALA-9242: Filter privileges before returning them to Sentry
..
Patch Set 9:
(17 comments)
Thanks everyone for your comments!
Some guide to view the new patch sets:
PS8: adds unit tests and fixed a bug about not returning server privileges when
listing URI privileges
PS9: fixes update behavior - this also needed Nodes to store
name->PrincipalPrivilege maps for values instead of sets of PrincipalPrivileges.
PS10: Fixes most of the review comments.
http://gerrit.cloudera.org:8080/#/c/15068/7/fe/src/main/java/org/apache/impala/catalog/Principal.java
File fe/src/main/java/org/apache/impala/catalog/Principal.java:
http://gerrit.cloudera.org:8080/#/c/15068/7/fe/src/main/java/org/apache/impala/catalog/Principal.java@104
PS7, Line 104:* Returns all privilege names for this principal, or an empty
set if no privileges are
> The doc comment is the same as for getPrivilegeNames() and does not mention
Done
http://gerrit.cloudera.org:8080/#/c/15068/7/fe/src/main/java/org/apache/impala/catalog/Principal.java@118
PS7, Line 118: Set results = new HashSet<>();
> May be I am overthinking this. Is there a race here? We are releasing the r
PrincipalPrivilege's fields are accessed at a lot of places in the code, so I
am hesitant about adding a comment about this here.
PrincipalPrivilege is practically treated as immutable - most of its fields are
included in the name (returned by getName()), which is also used as a key in
CatalogObjectCache, so changing any of these members would break our
assumptions about CatalogObjectCache regardless of thread safety. The only way
it should be changed is by replacing it with a new PrincipalPrivilege with the
same name (and higher catalog version). If this happens on another thread, the
old PrincipalPrivilege will be held by the current thread for the given
privilege check - my understanding is that this is the intended way to use
CatalogObjects.
http://gerrit.cloudera.org:8080/#/c/15068/7/fe/src/main/java/org/apache/impala/catalog/PrincipalPrivilegeTree.java
File fe/src/main/java/org/apache/impala/catalog/PrincipalPrivilegeTree.java:
http://gerrit.cloudera.org:8080/#/c/15068/7/fe/src/main/java/org/apache/impala/catalog/PrincipalPrivilegeTree.java@34
PS7, Line 34: Tree that allows efficient lookup
> Can we have a more detailed documentation of the class? Specifically, why i
I added more details in the comment.
http://gerrit.cloudera.org:8080/#/c/15068/7/fe/src/main/java/org/apache/impala/catalog/PrincipalPrivilegeTree.java@34
PS7, Line 34: for
> Isn't this 'for' superfluous?
Done
http://gerrit.cloudera.org:8080/#/c/15068/7/fe/src/main/java/org/apache/impala/catalog/PrincipalPrivilegeTree.java@37
PS7, Line 37: public class PrincipalPrivilegeTree {
> The implementation seems correct to me, though I haven't tried it yet, only
I added unit tests which (of course :/) found some bugs (wrong update logic +
not returning match server privileges when listing privileges for an URI).
http://gerrit.cloudera.org:8080/#/c/15068/7/fe/src/main/java/org/apache/impala/catalog/PrincipalPrivilegeTree.java@41
PS7, Line 41: Node tableRoot_ = new Node<>();
:
:// Contains URI privileges grouped by server name.
:Node uriRoot_ = new Node<>();
:
:// Contains server privileges grouped by server name.
:Node serverRoot_ = new Node<>();
> Can you clarify why we need to have 3 separate trees instead of just one?
Done
http://gerrit.cloudera.org:8080/#/c/15068/7/fe/src/main/java/org/apache/impala/catalog/PrincipalPrivilegeTree.java@105
PS7, Line 105: return path;
> Unclear what this means. Can you please clarify?
Done
http://gerrit.cloudera.org:8080/#/c/15068/7/fe/src/main/java/org/apache/impala/catalog/PrincipalPrivilegeTree.java@124
PS7, Line 124:
> Does server double up for storing uris too?
Yes, URIs are stored per server. Added some explanation in the class comment.
http://gerrit.cloudera.org:8080/#/c/15068/7/fe/src/main/java/org/apache/impala/catalog/PrincipalPrivilegeTree.java@125
PS7, Line 125: private List toPath() {
> if uri is not null, should you add it into path before return?
The URI string are intentionally not added to the path. Added some explanation
about this in the class comment.
http://gerrit.cloudera.org:8080/#/c/15068/7/fe/src/main/java/org/apache/impala/catalog/PrincipalPrivilegeTree.java@134
PS7, Line 134:
> Nit: object.
Done
http://gerrit.cloudera.org:8080/#/c/15068/7/fe/src/main/java/org/apache/impala/catalog/PrincipalPrivilegeTree.java@138
PS7, Line 138: Tree node that holds the privileges for a given objects (server,
database,
> probably unnecessary and can be removed.
I would prefer to leave it here - generally if there is a generic class, I
would assume that it is expec
[Impala-ASF-CR] IMPALA-9242: Filter privileges before returning them to Sentry
Hello Anonymous Coward (498), Vihang Karajgaonkar, Daniel Becker, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/15068 to look at the new patch set (#11). Change subject: IMPALA-9242: Filter privileges before returning them to Sentry .. IMPALA-9242: Filter privileges before returning them to Sentry This change implements the new FilteredPrivilegeCache, which adds functions for filtering privileges based on the authorizable and for returning Privileges directly instead of their String form. The filtering is based on server + db + table (or just server in case of URI privileges) to filter out the bulk of unrelated privileges. Efficient filtering is done by a new class PrincipalPrivilegeTree. It was tempting to reuse Sentry's TreePrivilegeCache, which has a very similar role, but it lacks a "remove" function that is needed to keep this index in sync with the CatalogObjectCache in Principal. I am also a bit concerned about the possible side effect of Sentry's interning of names in privileges - we try to avoid using String.intern() on massive amount of names in Impala. Other Changes: - Add the Sentry privilege name as member to PrincipalPrivileges. Note that the name was a member of TPrivilege till IMPALA-7616. Storing the name shouldn't consume much extra memory, as it is already stored as the key of the PrincipalPrivilege in CatalogObjectCache. Testing: - added unit tests based on Sentry / TestTreePrivilegeCache Change-Id: Iecd4281368d1c9fe88cfe850ea725cd68895712e --- M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationPolicy.java M fe/src/main/java/org/apache/impala/catalog/Principal.java M fe/src/main/java/org/apache/impala/catalog/PrincipalPrivilege.java A fe/src/main/java/org/apache/impala/catalog/PrincipalPrivilegeTree.java A fe/src/test/java/org/apache/impala/catalog/PrincipalPrivilegeTreeTest.java 5 files changed, 509 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/15068/11 -- To view, visit http://gerrit.cloudera.org:8080/15068 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iecd4281368d1c9fe88cfe850ea725cd68895712e Gerrit-Change-Number: 15068 Gerrit-PatchSet: 11 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Anonymous Coward (498) Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-9242: Filter privileges before returning them to Sentry
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15068 ) Change subject: IMPALA-9242: Filter privileges before returning them to Sentry .. Patch Set 10: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5182/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15068 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iecd4281368d1c9fe88cfe850ea725cd68895712e Gerrit-Change-Number: 15068 Gerrit-PatchSet: 10 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Anonymous Coward (498) Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Mon, 10 Feb 2020 15:58:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5904: Add full tsan option and fix several TSAN bugs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15116 ) Change subject: IMPALA-5904: Add full_tsan option and fix several TSAN bugs .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5309/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/15116 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3d7ef5c228afd5882e145e6f53885b355d6c25a0 Gerrit-Change-Number: 15116 Gerrit-PatchSet: 5 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 10 Feb 2020 15:59:40 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5904: Add full tsan option and fix several TSAN bugs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15116 ) Change subject: IMPALA-5904: Add full_tsan option and fix several TSAN bugs .. Patch Set 5: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/15116 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3d7ef5c228afd5882e145e6f53885b355d6c25a0 Gerrit-Change-Number: 15116 Gerrit-PatchSet: 5 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 10 Feb 2020 15:59:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9242: Filter privileges before returning them to Sentry
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15068 ) Change subject: IMPALA-9242: Filter privileges before returning them to Sentry .. Patch Set 11: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5183/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15068 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iecd4281368d1c9fe88cfe850ea725cd68895712e Gerrit-Change-Number: 15068 Gerrit-PatchSet: 11 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Anonymous Coward (498) Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Mon, 10 Feb 2020 16:18:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9242: Filter privileges before returning them to Sentry
Csaba Ringhofer has posted comments on this change. (
http://gerrit.cloudera.org:8080/15068 )
Change subject: IMPALA-9242: Filter privileges before returning them to Sentry
..
Patch Set 11:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/15068/7//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/15068/7//COMMIT_MSG@20
PS7, Line 20: String.intern()
> Curisous to know if we have any references like past JIRAs to show to Strin
This change added a "handmade" interner to Impala:
https://gerrit.cloudera.org/#/c/11158/
It mentions this post with Java 8 benchmarks:
https://shipilev.net/jvm-anatomy-park/10-string-intern/
The intention of the change was to reduce memory consumption by interning some
common strings, and it avoided String.intern() to be sure that no regression is
introduced.
http://gerrit.cloudera.org:8080/#/c/15068/7/fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationPolicy.java
File
fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationPolicy.java:
http://gerrit.cloudera.org:8080/#/c/15068/7/fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationPolicy.java@49
PS7, Line 49: public class SentryAuthorizationPolicy implements
FilteredPrivilegeCache {
> Do you think we should refactor this code such that we can revert back to P
I see 3 reason why the fallback could be useful:
1: to be able to work with a version of Sentry that doesn't support
FilteredPrivilegeCache
2: to be able to avoid the memory cost of the PrincipalPrivilegeTree
3: to be able to turn this off if it turns out to be buggy
I think that 1 is not a valid scenario, as Impala wouldn't even compile with
such a Sentry version. I expect this patch to be backported together with the
relevant Sentry changes.
2 makes sense as the tree does consume some extra memory for privileges, but I
assume that most users who have enough privileges for this to matter will also
have problems with the slow SHOW DATABASES / TABLES. (this is not necessarily
true, as Impala caches privileges for every user/role, while the speed is only
affected by the number of privileges of the current user and its roles).
I am not enthusiastic about 3, as adding an option could also introduce issues
(there were some Sentry bugs that only occurred if it had to fall back to
PrivilegeCache, see SENTRY-2545 and SENTRY-2549), so I am not sure that we
would really help here. Actually SENTRY-2549 is still not merged to ASF/master
Sentry, so falling back to PrivilegeCache is still buggy.
So I think that 2 is the only good reason, but if we want to spend time to
reduce the mem cost/privilege, I would prefer other ways, e.g. interning
database/table names in privileges (we already intern these string in other
CatalogObjects).
http://gerrit.cloudera.org:8080/#/c/15068/7/fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationPolicy.java@156
PS7, Line 156: filter
> It unclear why we are not storing the uri here? Can you clarify?
I added some explanation in PrincipalPrivilegeTree.
+1 reason is that URIs are also a bit more problematic than other objects due
to their case sensitivity.
http://gerrit.cloudera.org:8080/#/c/15068/7/fe/src/main/java/org/apache/impala/catalog/PrincipalPrivilegeTree.java
File fe/src/main/java/org/apache/impala/catalog/PrincipalPrivilegeTree.java:
http://gerrit.cloudera.org:8080/#/c/15068/7/fe/src/main/java/org/apache/impala/catalog/PrincipalPrivilegeTree.java@200
PS7, Line 200:
> Oops, I forgot this, I will do this in the next patch.
Done
--
To view, visit http://gerrit.cloudera.org:8080/15068
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iecd4281368d1c9fe88cfe850ea725cd68895712e
Gerrit-Change-Number: 15068
Gerrit-PatchSet: 11
Gerrit-Owner: Csaba Ringhofer
Gerrit-Reviewer: Anonymous Coward (498)
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Daniel Becker
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Vihang Karajgaonkar
Gerrit-Comment-Date: Mon, 10 Feb 2020 16:37:57 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9242: Filter privileges before returning them to Sentry
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15068 ) Change subject: IMPALA-9242: Filter privileges before returning them to Sentry .. Patch Set 11: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5310/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/15068 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iecd4281368d1c9fe88cfe850ea725cd68895712e Gerrit-Change-Number: 15068 Gerrit-PatchSet: 11 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Anonymous Coward (498) Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Mon, 10 Feb 2020 16:38:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4224: execute separate join builds fragments
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/14859 ) Change subject: IMPALA-4224: execute separate join builds fragments .. Patch Set 37: (3 comments) Thanks for the further details and comments. The change looks good to me, but it's massive and complex so it'd be good to involve more people. http://gerrit.cloudera.org:8080/#/c/14859/37/be/src/exec/join-builder.h File be/src/exec/join-builder.h: http://gerrit.cloudera.org:8080/#/c/14859/37/be/src/exec/join-builder.h@100 PS37, Line 100: /// | | |<| Thanks for the diagram, looks great! http://gerrit.cloudera.org:8080/#/c/14859/37/be/src/exec/join-builder.h@110 PS37, Line 110: CloseForProbe CloseFromProbe() http://gerrit.cloudera.org:8080/#/c/14859/35/fe/src/main/java/org/apache/impala/planner/Planner.java File fe/src/main/java/org/apache/impala/planner/Planner.java: http://gerrit.cloudera.org:8080/#/c/14859/35/fe/src/main/java/org/apache/impala/planner/Planner.java@283 PS35, Line 283: > The naming is a bit confusing - the single node plan here essentially means Oh I see, thanks for the explanation. -- To view, visit http://gerrit.cloudera.org:8080/14859 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4403c8e62d9c13854e7830602ee613f8efc80c58 Gerrit-Change-Number: 14859 Gerrit-PatchSet: 37 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 10 Feb 2020 16:40:28 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9199: Add support for single query retries on cluster membership changes
Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14824 ) Change subject: IMPALA-9199: Add support for single query retries on cluster membership changes .. Patch Set 5: (7 comments) Addressed some of the comments and filed a few follow up JIRAs. Going to start a doc to discuss the "transparent" part of this feature (e.g. the internal query ids). Working on doing some of the re-factoring you mentioned (e.g. adding another layer of abstraction on top of ClientRequestState). http://gerrit.cloudera.org:8080/#/c/14824/5//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/14824/5//COMMIT_MSG@26 PS5, Line 26: A query cannot be retried once any results from the original query : have been fetched, this is to prevent users from seeing incorrect results > Do we have any mechanism for a user to specify that they want their results Not yet, but planning to tackle that in a follow up JIRA: IMPALA-9225 http://gerrit.cloudera.org:8080/#/c/14824/5//COMMIT_MSG@41 PS5, Line 41: The Impala Web UI will list all retried queries as being in the : "RETRIED" state > This may be a reasonable approach, but I think we should think about the us Yeah, these are legitimate issues. We have few follow up JIRAs to tackle some of these problems: IMPALA-9229, IMPALA-9230, IMPALA-9213 and I just filed IMPALA-9364 as well - open to more suggestions about how to improve supportability. For this patch, I think it should be sufficient to add the original query id to the retried query profile, and vice versa. http://gerrit.cloudera.org:8080/#/c/14824/5/be/src/service/client-request-state-map.h File be/src/service/client-request-state-map.h: http://gerrit.cloudera.org:8080/#/c/14824/5/be/src/service/client-request-state-map.h@38 PS5, Line 38: bool overwrite = false > I don't think this is being used anywhere anymore. Done http://gerrit.cloudera.org:8080/#/c/14824/5/be/src/service/client-request-state.h File be/src/service/client-request-state.h: http://gerrit.cloudera.org:8080/#/c/14824/5/be/src/service/client-request-state.h@82 PS5, Line 82: RETRYING, RETRIED, UNKNOWN > I think this approach, and the way you have to save the previous state and Yeah, thats a good suggestion. Haven't looked into adding the ClientRequestState wrapper yet, but for now just split out the the RETRYING and RETRIED states into a `enum RetryState`. It actually simplifies the state management considerably, so thanks for the suggestion. http://gerrit.cloudera.org:8080/#/c/14824/5/be/src/service/impala-server.cc File be/src/service/impala-server.cc: http://gerrit.cloudera.org:8080/#/c/14824/5/be/src/service/impala-server.cc@632 PS5, Line 632: shared_ptr request_state = GetClientFacingRequestState(query_id); > Unless I'm missing something in this patch, this means that only the profil hmm not sure why I did that, changed back to GetClientRequestState http://gerrit.cloudera.org:8080/#/c/14824/5/be/src/service/impala-server.cc@920 PS5, Line 920: unique_ptr exec_request = make_unique(); > So I'm wondering if this patch would benefit from adding another layer of a I think that makes sense. IMO the ImpalaServer <--> ClientRequestState <--> Coordinator code needs some re-factoring in general. I wanted to some general re-factoring all this code (ImpalaServer, ClientRequestState, Coordinator) so I filed IMPALA-9370 as a follow up, but I think it makes sense to move some of the retry logic into its own class in this patch. I re-factored the BlockOnWait code so that it isn't duplicated across the hs2 and beeswax server. http://gerrit.cloudera.org:8080/#/c/14824/5/be/src/service/impala-server.cc@1025 PS5, Line 1025: query_ctx->query_id = UuidToQueryId(random_generator()()); > As mentioned before, I definitely think it would be awesome if we could cre I sort of did this in the previous patch update. I added a client_query_id_mapping_ that allows an optional mapping between query ids. It doesn't exactly create an "internal" id, but its cleaner that what I was doing before. I think I'm going to resurrect the design doc for this feature and write up some notes on how best to approach this. We can finalize the design there. The mixing of the hi/lo of the TUniqueId is an interesting idea. I'll be sure to include that. -- To view, visit http://gerrit.cloudera.org:8080/14824 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2e4a0e72a9bf8ec10b91639aefd81bef17886ddd Gerrit-Change-Number: 14824 Gerrit-PatchSet: 5 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Mon, 10 Feb 2020 16:50:40 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9199: Add support for single query retries on cluster membership changes
Hello Thomas Tauber-Marshall, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/14824
to look at the new patch set (#6).
Change subject: IMPALA-9199: Add support for single query retries on cluster
membership changes
..
IMPALA-9199: Add support for single query retries on cluster membership changes
Adds the core logic for transparently retrying queries that fail due to
cluster membership changes.
Query retries are triggered if (1) a node has been removed from the
cluster membership by a statestore update (rather than cancelling all
queries running on the leaving node, queries are retried), and (2) if a
query fails and as a result, blacklists a node. Both events are
considered cluster membership changes as they affect what nodes a query
will be scheduled on.
Implementation:
* Query retries are driven by a dedicated threadpool
* ImpalaServer::RetryQueryFromThreadPool implements the core logic to
actually retry a failed query.
* When a query is retried, the original query is cancelled, the new
query is created, registered, and started, and then the original query
is closed
* A query cannot be retried once any results from the original query
have been fetched, this is to prevent users from seeing incorrect results
Features:
* Retries are transparent to the user
* This is achieved by adding a mapping from failed query ids to the
query id of the retried query
* ImpalaServer uses this mapping in GetClientFacingRequestState
which is used to differentiate between "client facing" requests
for a ClientRequestState vs. internal requrets for a CRS
* Users can tell if a query is retried using runtime profiles and the
Impala Web UI
* "Impala Query Status" is a new field in runtime profiles that
displays the ClientRequestState execution state (which includes
the RETRYING and RETRIED states)
* The Impala Web UI will list all retried queries as being in the
"RETRIED" state
* Retried queries skip all fe/ planning, authorization, etc.
* This feature is configurable ('retry_failed_queries') and is off by
default
Refactoring:
* Changes the ClientRequestState so that it can take in an existing
TExecRequest
* This is required when retrying queries because the
TExecRequest of the failed query is copied and used for the
ClientRequestState of the retried query
* ClientRequestState::ExecState is extended with three new states:
RETRYING, RETRIED, and UNKNOWN.
Testing:
* Added integration tests in test_query_retries.py, these tests
consistently pass when run locally
* Ran exhaustive tests
* Ran exhaustive tests with 'retry_failed_queries' set to true, no
unexpected failures
TODO:
* Additional re-factoring / code cleanup
* Lots more documentation
Change-Id: I2e4a0e72a9bf8ec10b91639aefd81bef17886ddd
---
M be/src/runtime/coordinator.cc
M be/src/runtime/coordinator.h
M be/src/service/client-request-state.cc
M be/src/service/client-request-state.h
M be/src/service/impala-beeswax-server.cc
M be/src/service/impala-hs2-server.cc
M be/src/service/impala-http-handler.cc
M be/src/service/impala-server.cc
M be/src/service/impala-server.h
M be/src/service/query-options.cc
M be/src/service/query-options.h
A be/src/service/retry-work.h
M common/thrift/ImpalaInternalService.thrift
M common/thrift/ImpalaService.thrift
M common/thrift/generate_error_codes.py
A tests/custom_cluster/test_query_retries.py
16 files changed, 793 insertions(+), 168 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/24/14824/6
--
To view, visit http://gerrit.cloudera.org:8080/14824
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2e4a0e72a9bf8ec10b91639aefd81bef17886ddd
Gerrit-Change-Number: 14824
Gerrit-PatchSet: 6
Gerrit-Owner: Sahil Takiar
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Sahil Takiar
Gerrit-Reviewer: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-9199: Add support for single query retries on cluster membership changes
Hello Thomas Tauber-Marshall, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/14824
to look at the new patch set (#7).
Change subject: IMPALA-9199: Add support for single query retries on cluster
membership changes
..
IMPALA-9199: Add support for single query retries on cluster membership changes
Adds the core logic for transparently retrying queries that fail due to
cluster membership changes.
Query retries are triggered if (1) a node has been removed from the
cluster membership by a statestore update (rather than cancelling all
queries running on the leaving node, queries are retried), and (2) if a
query fails and as a result, blacklists a node. Both events are
considered cluster membership changes as they affect what nodes a query
will be scheduled on.
Implementation:
* Query retries are driven by a dedicated threadpool
* ImpalaServer::RetryQueryFromThreadPool implements the core logic to
actually retry a failed query.
* When a query is retried, the original query is cancelled, the new
query is created, registered, and started, and then the original query
is closed
* A query cannot be retried once any results from the original query
have been fetched, this is to prevent users from seeing incorrect results
Features:
* Retries are transparent to the user
* This is achieved by adding a mapping from failed query ids to the
query id of the retried query
* ImpalaServer uses this mapping in GetClientFacingRequestState
which is used to differentiate between "client facing" requests
for a ClientRequestState vs. internal requrets for a CRS
* Users can tell if a query is retried using runtime profiles and the
Impala Web UI
* "Impala Query Status" is a new field in runtime profiles that
displays the ClientRequestState execution state (which includes
the RETRYING and RETRIED states)
* The Impala Web UI will list all retried queries as being in the
"RETRIED" state
* Retried queries skip all fe/ planning, authorization, etc.
* This feature is configurable ('retry_failed_queries') and is off by
default
Refactoring:
* Changes the ClientRequestState so that it can take in an existing
TExecRequest
* This is required when retrying queries because the
TExecRequest of the failed query is copied and used for the
ClientRequestState of the retried query
* ClientRequestState::ExecState is extended with three new states:
RETRYING, RETRIED, and UNKNOWN.
Testing:
* Added integration tests in test_query_retries.py, these tests
consistently pass when run locally
* Ran exhaustive tests
* Ran exhaustive tests with 'retry_failed_queries' set to true, no
unexpected failures
TODO:
* Additional re-factoring / code cleanup
* Lots more documentation
Change-Id: I2e4a0e72a9bf8ec10b91639aefd81bef17886ddd
---
M be/src/runtime/coordinator.cc
M be/src/runtime/coordinator.h
M be/src/service/client-request-state.cc
M be/src/service/client-request-state.h
M be/src/service/impala-beeswax-server.cc
M be/src/service/impala-hs2-server.cc
M be/src/service/impala-http-handler.cc
M be/src/service/impala-server.cc
M be/src/service/impala-server.h
M be/src/service/query-options.cc
M be/src/service/query-options.h
A be/src/service/retry-work.h
M common/thrift/ImpalaInternalService.thrift
M common/thrift/ImpalaService.thrift
M common/thrift/generate_error_codes.py
A tests/custom_cluster/test_query_retries.py
16 files changed, 793 insertions(+), 168 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/24/14824/7
--
To view, visit http://gerrit.cloudera.org:8080/14824
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2e4a0e72a9bf8ec10b91639aefd81bef17886ddd
Gerrit-Change-Number: 14824
Gerrit-PatchSet: 7
Gerrit-Owner: Sahil Takiar
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Sahil Takiar
Gerrit-Reviewer: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-9199: Add support for single query retries on cluster membership changes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14824 ) Change subject: IMPALA-9199: Add support for single query retries on cluster membership changes .. Patch Set 6: (1 comment) http://gerrit.cloudera.org:8080/#/c/14824/6/be/src/service/impala-server.cc File be/src/service/impala-server.cc: http://gerrit.cloudera.org:8080/#/c/14824/6/be/src/service/impala-server.cc@1463 PS6, Line 1463: void ImpalaServer::BlockOnWait(shared_ptr* request_state, bool* timed_out, line too long (94 > 90) -- To view, visit http://gerrit.cloudera.org:8080/14824 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2e4a0e72a9bf8ec10b91639aefd81bef17886ddd Gerrit-Change-Number: 14824 Gerrit-PatchSet: 6 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Mon, 10 Feb 2020 16:51:03 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14782 ) Change subject: IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI .. Patch Set 9: (2 comments) http://gerrit.cloudera.org:8080/#/c/14782/9/tests/common/impala_test_suite.py File tests/common/impala_test_suite.py: http://gerrit.cloudera.org:8080/#/c/14782/9/tests/common/impala_test_suite.py@1123 PS9, Line 1123: : flake8: E501 line too long (91 > 90 characters) http://gerrit.cloudera.org:8080/#/c/14782/9/tests/common/impala_test_suite.py@1129 PS9, Line 1129: ) flake8: E999 SyntaxError: invalid syntax -- To view, visit http://gerrit.cloudera.org:8080/14782 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icf74ad06ce1c40fab4ce37de6b7ca78e3e520b43 Gerrit-Change-Number: 14782 Gerrit-PatchSet: 9 Gerrit-Owner: Vincent Tran Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vincent Tran Gerrit-Comment-Date: Mon, 10 Feb 2020 16:51:41 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI
Vincent Tran has uploaded a new patch set (#9). ( http://gerrit.cloudera.org:8080/14782 ) Change subject: IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI .. IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI This change appends the socket address (HOST:PORT) of the client who made the request to close a session or cancel a query from the coordinator's debug WebUI. Existing statuses: "Cancelled from Impala's debug web interface" "Session closed from Impala's debug web interface" New statuses: "Cancelled from Impala's debug web interface by client at :" "Session closed from Impala's debug web interface by client at :" Testing: -Verified visually that the status message is printed in the impalad log with the socket address when one cancels a query or closes a session. -Added a new e2e test to verify that the new status gets printed in runtime profile and coordinator log when a query is cancelled in this way. -Made log asserts more robust by adding a timeout/wait value. Change-Id: Icf74ad06ce1c40fab4ce37de6b7ca78e3e520b43 --- M be/src/kudu/util/web_callback_registry.h M be/src/service/impala-http-handler.cc M be/src/util/webserver.cc M tests/common/impala_test_suite.py M tests/observability/test_log_fragments.py M tests/webserver/test_web_pages.py 6 files changed, 76 insertions(+), 30 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/82/14782/9 -- To view, visit http://gerrit.cloudera.org:8080/14782 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icf74ad06ce1c40fab4ce37de6b7ca78e3e520b43 Gerrit-Change-Number: 14782 Gerrit-PatchSet: 9 Gerrit-Owner: Vincent Tran Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vincent Tran
[Impala-ASF-CR] IMPALA-9199: Add support for single query retries on cluster membership changes
Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14824 ) Change subject: IMPALA-9199: Add support for single query retries on cluster membership changes .. Patch Set 6: (1 comment) http://gerrit.cloudera.org:8080/#/c/14824/6/be/src/service/impala-server.cc File be/src/service/impala-server.cc: http://gerrit.cloudera.org:8080/#/c/14824/6/be/src/service/impala-server.cc@1463 PS6, Line 1463: void ImpalaServer::BlockOnWait(shared_ptr* request_state, bool* timed_out, > line too long (94 > 90) Done -- To view, visit http://gerrit.cloudera.org:8080/14824 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2e4a0e72a9bf8ec10b91639aefd81bef17886ddd Gerrit-Change-Number: 14824 Gerrit-PatchSet: 6 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Mon, 10 Feb 2020 16:51:50 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI
Vincent Tran has uploaded a new patch set (#10). ( http://gerrit.cloudera.org:8080/14782 ) Change subject: IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI .. IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI This change appends the socket address (HOST:PORT) of the client who made the request to close a session or cancel a query from the coordinator's debug WebUI. Existing statuses: "Cancelled from Impala's debug web interface" "Session closed from Impala's debug web interface" New statuses: "Cancelled from Impala's debug web interface by client at :" "Session closed from Impala's debug web interface by client at :" Testing: -Verified visually that the status message is printed in the impalad log with the socket address when one cancels a query or closes a session. -Added a new e2e test to verify that the new status gets printed in runtime profile and coordinator log when a query is cancelled in this way. -Made log asserts more robust by adding a timeout/wait value. Change-Id: Icf74ad06ce1c40fab4ce37de6b7ca78e3e520b43 --- M be/src/kudu/util/web_callback_registry.h M be/src/service/impala-http-handler.cc M be/src/util/webserver.cc M tests/common/impala_test_suite.py M tests/observability/test_log_fragments.py M tests/webserver/test_web_pages.py 6 files changed, 77 insertions(+), 30 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/82/14782/10 -- To view, visit http://gerrit.cloudera.org:8080/14782 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icf74ad06ce1c40fab4ce37de6b7ca78e3e520b43 Gerrit-Change-Number: 14782 Gerrit-PatchSet: 10 Gerrit-Owner: Vincent Tran Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vincent Tran
[Impala-ASF-CR] IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14782 ) Change subject: IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI .. Patch Set 10: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14782 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icf74ad06ce1c40fab4ce37de6b7ca78e3e520b43 Gerrit-Change-Number: 14782 Gerrit-PatchSet: 10 Gerrit-Owner: Vincent Tran Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vincent Tran Gerrit-Comment-Date: Mon, 10 Feb 2020 17:27:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14782 ) Change subject: IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI .. Patch Set 11: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14782 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icf74ad06ce1c40fab4ce37de6b7ca78e3e520b43 Gerrit-Change-Number: 14782 Gerrit-PatchSet: 11 Gerrit-Owner: Vincent Tran Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vincent Tran Gerrit-Comment-Date: Mon, 10 Feb 2020 17:27:20 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14782 ) Change subject: IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI .. Patch Set 11: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5311/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/14782 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icf74ad06ce1c40fab4ce37de6b7ca78e3e520b43 Gerrit-Change-Number: 14782 Gerrit-PatchSet: 11 Gerrit-Owner: Vincent Tran Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vincent Tran Gerrit-Comment-Date: Mon, 10 Feb 2020 17:27:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8852: Skip short-circuit config check for coordinator-only mode
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/15173 ) Change subject: IMPALA-8852: Skip short-circuit config check for coordinator-only mode .. Patch Set 4: Code-Review+2 LGTM, thanks -- To view, visit http://gerrit.cloudera.org:8080/15173 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I373d4037f4cee203322a398b77b75810ba708bb5 Gerrit-Change-Number: 15173 Gerrit-PatchSet: 4 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Mon, 10 Feb 2020 17:28:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8852: Skip short-circuit config check for coordinator-only mode
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15173 ) Change subject: IMPALA-8852: Skip short-circuit config check for coordinator-only mode .. Patch Set 5: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/15173 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I373d4037f4cee203322a398b77b75810ba708bb5 Gerrit-Change-Number: 15173 Gerrit-PatchSet: 5 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Mon, 10 Feb 2020 17:30:16 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8852: Skip short-circuit config check for coordinator-only mode
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15173 ) Change subject: IMPALA-8852: Skip short-circuit config check for coordinator-only mode .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5312/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/15173 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I373d4037f4cee203322a398b77b75810ba708bb5 Gerrit-Change-Number: 15173 Gerrit-PatchSet: 5 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Mon, 10 Feb 2020 17:30:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9199: Add support for single query retries on cluster membership changes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14824 ) Change subject: IMPALA-9199: Add support for single query retries on cluster membership changes .. Patch Set 6: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/5184/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/14824 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2e4a0e72a9bf8ec10b91639aefd81bef17886ddd Gerrit-Change-Number: 14824 Gerrit-PatchSet: 6 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Mon, 10 Feb 2020 17:34:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14782 ) Change subject: IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI .. Patch Set 9: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/5185/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/14782 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icf74ad06ce1c40fab4ce37de6b7ca78e3e520b43 Gerrit-Change-Number: 14782 Gerrit-PatchSet: 9 Gerrit-Owner: Vincent Tran Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vincent Tran Gerrit-Comment-Date: Mon, 10 Feb 2020 17:35:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9199: Add support for single query retries on cluster membership changes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14824 ) Change subject: IMPALA-9199: Add support for single query retries on cluster membership changes .. Patch Set 7: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5186/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14824 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2e4a0e72a9bf8ec10b91639aefd81bef17886ddd Gerrit-Change-Number: 14824 Gerrit-PatchSet: 7 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Mon, 10 Feb 2020 17:36:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14782 ) Change subject: IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI .. Patch Set 10: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5187/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14782 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icf74ad06ce1c40fab4ce37de6b7ca78e3e520b43 Gerrit-Change-Number: 14782 Gerrit-PatchSet: 10 Gerrit-Owner: Vincent Tran Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vincent Tran Gerrit-Comment-Date: Mon, 10 Feb 2020 17:45:30 + Gerrit-HasComments: No
[native-toolchain-CR] IMPALA-9279: part 2: Bump Kudu version to 5c610bf40
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/15192 ) Change subject: IMPALA-9279: part 2: Bump Kudu version to 5c610bf40 .. Patch Set 1: Code-Review+2 Looks good -- To view, visit http://gerrit.cloudera.org:8080/15192 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3ff92cc5e1de220a4c140cf6c0117b5fa1e89226 Gerrit-Change-Number: 15192 Gerrit-PatchSet: 1 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Mon, 10 Feb 2020 17:55:26 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP: Asynchronous code generation
Zoltan Borok-Nagy has posted comments on this change. (
http://gerrit.cloudera.org:8080/15105 )
Change subject: WIP: Asynchronous code generation
..
Patch Set 5:
(14 comments)
http://gerrit.cloudera.org:8080/#/c/15105/6/be/src/benchmarks/hash-benchmark.cc
File be/src/benchmarks/hash-benchmark.cc:
http://gerrit.cloudera.org:8080/#/c/15105/6/be/src/benchmarks/hash-benchmark.cc@530
PS6, Line 530: .load()
std::atomics have user-defined conversions to T, maybe you could also add to
make it more convenient.
https://en.cppreference.com/w/cpp/atomic/atomic/operator_T
http://gerrit.cloudera.org:8080/#/c/15105/1/be/src/codegen/codegen-fn-ptr.h
File be/src/codegen/codegen-fn-ptr.h:
http://gerrit.cloudera.org:8080/#/c/15105/1/be/src/codegen/codegen-fn-ptr.h@30
PS1, Line 30: rn ptr_.load(mem_order_);
> I think relaxed is correct for the reasons you mention.
+1 for acq/rel because I think it has the perfect balance between correctness
and performance. It provides quite strong guarantees and x86 supports it
without any additional cpu operations unlike sequential consistency.
http://gerrit.cloudera.org:8080/#/c/15105/6/be/src/codegen/llvm-codegen.h
File be/src/codegen/llvm-codegen.h:
http://gerrit.cloudera.org:8080/#/c/15105/6/be/src/codegen/llvm-codegen.h@345
PS6, Line 345: `
We usually use ' and not `.
http://gerrit.cloudera.org:8080/#/c/15105/6/be/src/codegen/llvm-codegen.h@352
PS6, Line 352: correspinging
corresponding
http://gerrit.cloudera.org:8080/#/c/15105/6/be/src/codegen/llvm-codegen.cc
File be/src/codegen/llvm-codegen.cc:
http://gerrit.cloudera.org:8080/#/c/15105/6/be/src/codegen/llvm-codegen.cc@477
PS6, Line 477: if (async_compile_thread_ != nullptr) {
: async_compile_thread_->Join();
: }
nit: fits single line
http://gerrit.cloudera.org:8080/#/c/15105/6/be/src/exec/grouping-aggregator.h
File be/src/exec/grouping-aggregator.h:
http://gerrit.cloudera.org:8080/#/c/15105/6/be/src/exec/grouping-aggregator.h@167
PS6, Line 167: Function type: AddBatchStreamingImplFn.
nit: Unnecessary comment.
http://gerrit.cloudera.org:8080/#/c/15105/6/be/src/exec/hdfs-scanner.h
File be/src/exec/hdfs-scanner.h:
http://gerrit.cloudera.org:8080/#/c/15105/6/be/src/exec/hdfs-scanner.h@462
PS6, Line 462: odegen is either disabled or not ready yet.
Would decltype(auto) work? You have multiple return stmts, but the return types
must be the same.
http://gerrit.cloudera.org:8080/#/c/15105/6/be/src/exec/hdfs-scanner.h@465
PS6, Line 465:
You need 'Args&&' if you want perfect forwarding.
http://gerrit.cloudera.org:8080/#/c/15105/6/be/src/exec/hdfs-scanner.h@487
PS6, Line 487: Status ScannerDebugAction() WARN_UNUSED_RESULT {
nit: please add short comment
http://gerrit.cloudera.org:8080/#/c/15105/6/be/src/exec/partitioned-hash-join-builder.h
File be/src/exec/partitioned-hash-join-builder.h:
http://gerrit.cloudera.org:8080/#/c/15105/6/be/src/exec/partitioned-hash-join-builder.h@721
PS6, Line 721: /// Function type: ProcessBuildBatchFn
nit: unnecessary comment
http://gerrit.cloudera.org:8080/#/c/15105/6/be/src/exec/partitioned-hash-join-builder.h@728
PS6, Line 728: /// Function type: InsertBatchFn.
unnecessary comment
http://gerrit.cloudera.org:8080/#/c/15105/6/be/src/exec/topn-node.h
File be/src/exec/topn-node.h:
http://gerrit.cloudera.org:8080/#/c/15105/6/be/src/exec/topn-node.h@140
PS6, Line 140: Function type: InsertBatchFn
unnecessary comment
http://gerrit.cloudera.org:8080/#/c/15105/6/be/src/exprs/scalar-expr.inline.h
File be/src/exprs/scalar-expr.inline.h:
http://gerrit.cloudera.org:8080/#/c/15105/6/be/src/exprs/scalar-expr.inline.h@48
PS6, Line 48:
nit: missing indentation, and maybe put the = sign to the prev line
http://gerrit.cloudera.org:8080/#/c/15105/6/be/src/runtime/runtime-state.h
File be/src/runtime/runtime-state.h:
http://gerrit.cloudera.org:8080/#/c/15105/6/be/src/runtime/runtime-state.h@148
PS6, Line 148: if the expression is not interpretable
I'm just wondering what expressions are not interpretable but can be codegen'd?
--
To view, visit http://gerrit.cloudera.org:8080/15105
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia7cbfa7c6734dcf03641629429057d6a4194aa6b
Gerrit-Change-Number: 15105
Gerrit-PatchSet: 5
Gerrit-Owner: Daniel Becker
Gerrit-Reviewer: Bikramjeet Vig
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Daniel Becker
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong
Gerrit-Reviewer: Zoltan Borok-Nagy
Gerrit-Comment-Date: Mon, 10 Feb 2020 18:00:23 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] Reapply IMPALA-9128: part 2: dump traces for slow RPCs
Tim Armstrong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15198 Change subject: Reapply IMPALA-9128: part 2: dump traces for slow RPCs .. Reapply IMPALA-9128: part 2: dump traces for slow RPCs This change was accidentally reverted in the KRPC rebase. It will be upstreamed to Kudu as KUDU-2996. Change-Id: Ic6800b68f61e84420e138bfad67a40be3796b4df --- M be/src/kudu/rpc/rpcz_store.cc 1 file changed, 5 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/98/15198/1 -- To view, visit http://gerrit.cloudera.org:8080/15198 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ic6800b68f61e84420e138bfad67a40be3796b4df Gerrit-Change-Number: 15198 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong
[Impala-ASF-CR] IMPALA-8778: Support Apache Hudi Read Optimized Table
Yanjia Gary Li has uploaded a new patch set (#23). ( http://gerrit.cloudera.org:8080/14711 ) Change subject: IMPALA-8778: Support Apache Hudi Read Optimized Table .. IMPALA-8778: Support Apache Hudi Read Optimized Table Hudi Read Optimized Table contains multiple versions of parquet files, in order to load the table correctly, Impala needs to recognize Hudi Read Optimized Table as a HdfsTable and load the latest version of the file using HoodieROTablePathFilter. Tests - Unit test for Hudi in FileMetadataLoader - Create table tests in functional_schema_template.sql - Query tests in hudi-parquet.test Change-Id: I65e146b347714df32fe968409ef2dde1f6a25cdf --- M be/src/service/query-options-test.cc M bin/impala-config.sh M bin/rat_exclude_files.txt M common/thrift/CatalogObjects.thrift M fe/pom.xml M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java M fe/src/main/java/org/apache/impala/catalog/FileMetadataLoader.java M fe/src/main/java/org/apache/impala/catalog/HdfsFileFormat.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java A fe/src/main/java/org/apache/impala/util/HudiUtil.java M fe/src/main/jflex/sql-scanner.flex M fe/src/test/java/org/apache/impala/catalog/FileMetadataLoaderTest.java M impala-parent/pom.xml M testdata/bin/generate-schema-statements.py M testdata/data/README A testdata/data/hudi_parquet/.hoodie/20200210090610.clean A testdata/data/hudi_parquet/.hoodie/20200210090610.clean.inflight A testdata/data/hudi_parquet/.hoodie/20200210090610.clean.requested A testdata/data/hudi_parquet/.hoodie/20200210090610.commit A testdata/data/hudi_parquet/.hoodie/20200210090610.commit.requested A testdata/data/hudi_parquet/.hoodie/20200210090610.inflight A testdata/data/hudi_parquet/.hoodie/20200210090618.clean A testdata/data/hudi_parquet/.hoodie/20200210090618.clean.inflight A testdata/data/hudi_parquet/.hoodie/20200210090618.clean.requested A testdata/data/hudi_parquet/.hoodie/20200210090618.commit A testdata/data/hudi_parquet/.hoodie/20200210090618.commit.requested A testdata/data/hudi_parquet/.hoodie/20200210090618.inflight A testdata/data/hudi_parquet/.hoodie/hoodie.properties A testdata/data/hudi_parquet/year=2015/month=03/day=16/.hoodie_partition_metadata A testdata/data/hudi_parquet/year=2015/month=03/day=16/5f541af5-ca07-4329-ad8c-40fa9b353f35-0_1-70-118_20200210090610.parquet A testdata/data/hudi_parquet/year=2015/month=03/day=16/5f541af5-ca07-4329-ad8c-40fa9b353f35-0_2-103-391_20200210090618.parquet A testdata/data/hudi_parquet/year=2015/month=03/day=17/.hoodie_partition_metadata A testdata/data/hudi_parquet/year=2015/month=03/day=17/675e035d-c146-4658-9404-fe590e296d80-0_0-103-389_20200210090618.parquet A testdata/data/hudi_parquet/year=2015/month=03/day=17/675e035d-c146-4658-9404-fe590e296d80-0_0-70-117_20200210090610.parquet A testdata/data/hudi_parquet/year=2016/month=03/day=15/.hoodie_partition_metadata A testdata/data/hudi_parquet/year=2016/month=03/day=15/940359ee-cc79-4974-8a2a-5d133a81a3fd-0_1-103-390_20200210090618.parquet A testdata/data/hudi_parquet/year=2016/month=03/day=15/940359ee-cc79-4974-8a2a-5d133a81a3fd-0_2-70-119_20200210090610.parquet M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv A testdata/workloads/functional-query/queries/QueryTest/hudi-parquet.test M testdata/workloads/functional-query/queries/QueryTest/set.test M tests/query_test/test_scanners.py 44 files changed, 626 insertions(+), 41 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/11/14711/23 -- To view, visit http://gerrit.cloudera.org:8080/14711 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I65e146b347714df32fe968409ef2dde1f6a25cdf Gerrit-Change-Number: 14711 Gerrit-PatchSet: 23 Gerrit-Owner: Yanjia Gary Li Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Yanjia Gary Li Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-5308: Resolve confusing Kudu SHOW TABLE STATS output
Tamas Mate has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15199 Change subject: IMPALA-5308: Resolve confusing Kudu SHOW TABLE STATS output .. IMPALA-5308: Resolve confusing Kudu SHOW TABLE STATS output This change modifies the output of the SHOW STABLE STATS for Kudu tables: - #Rows column is moved after partition key details - The values of #Rows column changed from '-1' to 'N/A' - 'Total' row is appended after parititions similar to HdfsTable Example output can be seen in the doc changes. Testing: kudu_stats.test is modified to verify the new style and 'Total" row. Change-Id: Ice4b8df65f0a53fe14b8fbe35d82c9887ab9a041 --- M docs/topics/impala_compute_stats.xml M docs/topics/impala_show.xml M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java M testdata/workloads/functional-query/queries/QueryTest/kudu_stats.test 4 files changed, 70 insertions(+), 41 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/15199/2 -- To view, visit http://gerrit.cloudera.org:8080/15199 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ice4b8df65f0a53fe14b8fbe35d82c9887ab9a041 Gerrit-Change-Number: 15199 Gerrit-PatchSet: 2 Gerrit-Owner: Tamas Mate
[Impala-ASF-CR] IMPALA-7002: Throw AuthorizationException when user accessing non-existent table/database in CTE without any privilege.
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/15123 ) Change subject: IMPALA-7002: Throw AuthorizationException when user accessing non-existent table/database in CTE without any privilege. .. Patch Set 6: Code-Review+2 (4 comments) I can start GVO build once these nits are addressed http://gerrit.cloudera.org:8080/#/c/15123/6//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/15123/6//COMMIT_MSG@7 PS6, Line 7: accessing nit: accesses http://gerrit.cloudera.org:8080/#/c/15123/6//COMMIT_MSG@8 PS6, Line 8: any privilege nit: required privileges http://gerrit.cloudera.org:8080/#/c/15123/6//COMMIT_MSG@10 PS6, Line 10: Traced the issue and found that privilege requests collected during : analysis were lost in WithClause::analyze function when analysis : function throw AnalysisException. This caused authorization been : skipped and returned analysis error, instead of authorization error. : This patch register the privilege requests made from root analyzer : to the input analyzer in WithClause::analyze function regardless of : analysis exception. Currently if a user without required privileges tries to access a non-existent table, then impala returns an analysis exception instead of authorization exception. This happens because during analysis of the with clause, the authorization request does not get registered due to analysis exception being thrown before it. This patch makes sure that those requests get registered regardless. http://gerrit.cloudera.org:8080/#/c/15123/5/fe/src/main/java/org/apache/impala/analysis/WithClause.java File fe/src/main/java/org/apache/impala/analysis/WithClause.java: http://gerrit.cloudera.org:8080/#/c/15123/5/fe/src/main/java/org/apache/impala/analysis/WithClause.java@95 PS5, Line 95: // Don't need catch block since the exception will be handled by the caller. > nit: no need for this comment -- To view, visit http://gerrit.cloudera.org:8080/15123 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia6b657a7147a136198a9a97a679c9131ee814577 Gerrit-Change-Number: 15123 Gerrit-PatchSet: 6 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 10 Feb 2020 18:29:25 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4224: execute separate join builds fragments
Hello Zoltan Borok-Nagy, Csaba Ringhofer, Bikramjeet Vig, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14859 to look at the new patch set (#38). Change subject: IMPALA-4224: execute separate join builds fragments .. IMPALA-4224: execute separate join builds fragments This enables parallel plans with the join build in a separate fragment and fixes all of the ensuing fallout. After this change, mt_dop plans with joins have separate build fragments. There is still a 1:1 relationship between join nodes and builders, so the builders are only accessed by the join node's thread after it is handed off. This lets us defer the work required to make PhjBuilder and NljBuilder safe to be shared between nodes. Planner changes: * Combined the parallel and distributed planning code paths. * Misc fixes to generate reasonable thrift structures in the query exec requests, i.e. containing the right nodes. * Fixes to resource calculations for the separate build plans. ** Calculate separate join/build resource consumption. ** Simplified the resource estimation by calculating resource consumption for each fragment separately, and assuming that all fragments hit their peak resource consumption at the same time. IMPALA-9255 is the follow-on to make the resource estimation more accurate. Scheduler changes: * Various fixes to handle multiple TPlanExecInfos correctly, which are generated by the planner for the different cohorts. * Add logic to colocate build fragments with parent fragments. Runtime filter changes: * Build sinks now produce runtime filters, which required planner and coordinator fixes to handle. DataSink changes: * Close the input plan tree before calling FlushFinal() to release resources. This depends on Send() not holding onto references to input batches, which was true except for NljBuilder. This invariant is documented. Join builder changes: * Add a common base class for PhjBuilder and NljBuilder with functions to handle synchronisation with the join node. * Close plan tree earlier in FragmentInstanceState::Exec() so that peak resource requirements are lower. * The NLJ always copies input batches, so that it can close its input tree. JoinNode changes: * Join node blocks waiting for build-side to be ready, then eventually signals that it's done, allowing the builder to be cleaned up. * NLJ and PHJ nodes handle both the integrated builder and the external builder. There is a 1:1 relationship between the node and the builder, so we don't deal with thread safety yet. * Buffer reservations are transferred between the builder and join node when running with the separate builder. This is not really necessary right now, since it is all single-threaded, but will be important for the shared broadcast. - The builder transfers memory for probe buffers to the join node at the end of each build phase. - At end of each probe phase, reservation needs to be handed back to builder (or released). ExecSummary changes: * The summary logic was modified to handle connecting fragments via join builds. The logic is an extension of what was used for exchanges. Testing: * Enable --unlock_mt_dop for end-to-end tests * Migrate some tests to run as part of end-to-end tests instead of custom cluster. * Add mt_dop dimension to various end-to-end tests to provide coverage of join queries, spill-to-disk and cancellation. * Ran a single node TPC-H and TPC-DS stress test with mt_dop=0 and mt_dop=4. Perf: * Ran TPC-H scale factor 30 locally with mt_dop=0. No significant change. Change-Id: I4403c8e62d9c13854e7830602ee613f8efc80c58 --- M be/src/exec/CMakeLists.txt M be/src/exec/blocking-join-node.cc M be/src/exec/blocking-join-node.h M be/src/exec/data-sink.cc M be/src/exec/data-sink.h M be/src/exec/exec-node.h A be/src/exec/join-builder.cc A be/src/exec/join-builder.h M be/src/exec/nested-loop-join-builder.cc M be/src/exec/nested-loop-join-builder.h M be/src/exec/nested-loop-join-node.cc M be/src/exec/nested-loop-join-node.h M be/src/exec/partitioned-hash-join-builder.cc M be/src/exec/partitioned-hash-join-builder.h M be/src/exec/partitioned-hash-join-node.cc M be/src/exec/partitioned-hash-join-node.h M be/src/runtime/bufferpool/buffer-pool-internal.h M be/src/runtime/bufferpool/buffer-pool-test.cc M be/src/runtime/bufferpool/buffer-pool.cc M be/src/runtime/bufferpool/buffer-pool.h M be/src/runtime/coordinator-backend-state.cc M be/src/runtime/coordinator.cc M be/src/runtime/fragment-instance-state.cc M be/src/runtime/fragment-instance-state.h M be/src/runtime/initial-reservations.cc M be/src/runtime/row-batch.cc M be/src/runtime/runtime-state.cc M be/src/runtime/runtime-state.h M be/src/runtime/spillable-row-batch-queue.h M be/src/util/summary-util.cc M bin/run-all-tests.sh M common/thrift/DataSinks.thrift M common/thrift/ExecStats.thrif
[Impala-ASF-CR] IMPALA-9287: Add support for embedded HMS in CDP builds
Vihang Karajgaonkar has posted comments on this change. (
http://gerrit.cloudera.org:8080/15057 )
Change subject: IMPALA-9287: Add support for embedded HMS in CDP builds
..
Patch Set 12:
(2 comments)
> > (1 comment)
>
> The error is: cannot found class PartitionExpressionForMetastore. I
> add class one by one to keep pom files minimal, all including four
> classed.
Ah I see. I think this can be fixed by setting metastore.expression.proxy to
org.apache.hadoop.hive.metastore.DefaultPartitionExpressionProxy in the
hive-site.xml.py if hive_major_version >=3 here:
https://github.com/apache/impala/blob/master/fe/src/test/resources/hive-site.xml.py#L93
The PartitionExpressionProxy API is only usable by Hive since it uses some of
its internal classes which are unavailable to non-hive applications. Refer
https://cwiki.apache.org/confluence/display/Hive/AdminManual+Metastore+3.0+Administration
standalone mode for more details.
http://gerrit.cloudera.org:8080/#/c/15057/12/fe/pom.xml
File fe/pom.xml:
http://gerrit.cloudera.org:8080/#/c/15057/12/fe/pom.xml@1065
PS12, Line 1065: runtime
I would avoid making any changes to hive-2 profile since they seem unnecessary.
http://gerrit.cloudera.org:8080/#/c/15057/12/fe/pom.xml@1235
PS12, Line 1235: ${hive.version}
This dependency can be changed to a runtime scope here for hive-3 profile only
since we don't want to depend on it for compilation.
runtime
--
To view, visit http://gerrit.cloudera.org:8080/15057
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ibc7d7e30cd560d43bb707dec54f4494355809f66
Gerrit-Change-Number: 15057
Gerrit-PatchSet: 12
Gerrit-Owner: wangsheng
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Quanlong Huang
Gerrit-Reviewer: Vihang Karajgaonkar
Gerrit-Reviewer: wangsheng
Gerrit-Comment-Date: Mon, 10 Feb 2020 18:45:27 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9287: Add support for embedded HMS in CDP builds
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/15057 ) Change subject: IMPALA-9287: Add support for embedded HMS in CDP builds .. Patch Set 12: (1 comment) http://gerrit.cloudera.org:8080/#/c/15057/12/shaded-deps/pom.xml File shaded-deps/pom.xml: http://gerrit.cloudera.org:8080/#/c/15057/12/shaded-deps/pom.xml@99 PS12, Line 99: See my comment earlier on PartitionExpressionProxy configurations. I think these are not necessary if we change the default configuration values for metastore. -- To view, visit http://gerrit.cloudera.org:8080/15057 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibc7d7e30cd560d43bb707dec54f4494355809f66 Gerrit-Change-Number: 15057 Gerrit-PatchSet: 12 Gerrit-Owner: wangsheng Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: wangsheng Gerrit-Comment-Date: Mon, 10 Feb 2020 18:46:37 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4224: execute separate join builds fragments
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14859 ) Change subject: IMPALA-4224: execute separate join builds fragments .. Patch Set 37: (1 comment) http://gerrit.cloudera.org:8080/#/c/14859/37/be/src/exec/join-builder.h File be/src/exec/join-builder.h: http://gerrit.cloudera.org:8080/#/c/14859/37/be/src/exec/join-builder.h@110 PS37, Line 110: CloseForProbe > CloseFromProbe() Done -- To view, visit http://gerrit.cloudera.org:8080/14859 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4403c8e62d9c13854e7830602ee613f8efc80c58 Gerrit-Change-Number: 14859 Gerrit-PatchSet: 37 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 10 Feb 2020 18:54:10 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Reapply IMPALA-9128: part 2: dump traces for slow RPCs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15198 ) Change subject: Reapply IMPALA-9128: part 2: dump traces for slow RPCs .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5188/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15198 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic6800b68f61e84420e138bfad67a40be3796b4df Gerrit-Change-Number: 15198 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 10 Feb 2020 18:59:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8778: Support Apache Hudi Read Optimized Table
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14711 ) Change subject: IMPALA-8778: Support Apache Hudi Read Optimized Table .. Patch Set 23: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5189/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14711 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I65e146b347714df32fe968409ef2dde1f6a25cdf Gerrit-Change-Number: 14711 Gerrit-PatchSet: 23 Gerrit-Owner: Yanjia Gary Li Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Yanjia Gary Li Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 10 Feb 2020 19:06:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7002: Throw AuthorizationException when user accessing non-existent table/database in CTE without any privilege.
Wenzhe Zhou has posted comments on this change. (
http://gerrit.cloudera.org:8080/15123 )
Change subject: IMPALA-7002: Throw AuthorizationException when user accessing
non-existent table/database in CTE without any privilege.
..
Patch Set 6:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/15123/6//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/15123/6//COMMIT_MSG@7
PS6, Line 7: accessing
> nit: accesses
will fix it
http://gerrit.cloudera.org:8080/#/c/15123/6//COMMIT_MSG@8
PS6, Line 8: any privilege
> nit: required privileges
will fix it.
http://gerrit.cloudera.org:8080/#/c/15123/6//COMMIT_MSG@10
PS6, Line 10: Traced the issue and found that privilege requests collected
during
: analysis were lost in WithClause::analyze function when analysis
: function throw AnalysisException. This caused authorization been
: skipped and returned analysis error, instead of authorization
error.
: This patch register the privilege requests made from root analyzer
: to the input analyzer in WithClause::analyze function regardless
of
: analysis exception.
> Currently if a user without required privileges tries to access a non-exist
will update as suggested
http://gerrit.cloudera.org:8080/#/c/15123/6/fe/src/main/java/org/apache/impala/analysis/WithClause.java
File fe/src/main/java/org/apache/impala/analysis/WithClause.java:
http://gerrit.cloudera.org:8080/#/c/15123/6/fe/src/main/java/org/apache/impala/analysis/WithClause.java@96
PS6, Line 96: finally {
: // Register all privilege requests made from the root
analyzer to the input
: // analyzer so that caller could do authorization for all
the requests collected
: // during analysis and report an authorization error over
an analysis error.
: // We should not accidentally reveal the non-existence of a
database/table if
: // the user is not authorized.
: for (PrivilegeRequest req :
withClauseAnalyzer.getPrivilegeReqs()) {
: analyzer.registerPrivReq(req);
: }
: }
> Thanks to Wenzhe for the detailed explanation! I have checked the code path
So the code change is safe.
--
To view, visit http://gerrit.cloudera.org:8080/15123
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia6b657a7147a136198a9a97a679c9131ee814577
Gerrit-Change-Number: 15123
Gerrit-PatchSet: 6
Gerrit-Owner: Wenzhe Zhou
Gerrit-Reviewer: Bikramjeet Vig
Gerrit-Reviewer: Fang-Yu Rao
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Wenzhe Zhou
Gerrit-Comment-Date: Mon, 10 Feb 2020 19:08:05 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7002: Throw AuthorizationException when user accesss non-existent table/database in CTE without required privileges.
Wenzhe Zhou has uploaded a new patch set (#7). ( http://gerrit.cloudera.org:8080/15123 ) Change subject: IMPALA-7002: Throw AuthorizationException when user accesss non-existent table/database in CTE without required privileges. .. IMPALA-7002: Throw AuthorizationException when user accesss non-existent table/database in CTE without required privileges. Currently if a user without required privileges tries to access a non-existent database or table, then impala returns an analysis exception instead of authorization exception. This happens because during analysis of the with clause, the authorization request does not get registered due to analysis exception being thrown before it. This patch makes sure that those requests get registered regardless. Testing: - Manual test: - ran CTE with non-existent table/database in impala-shell without privilege, verified that it results in AuthorizationException. - ran CTE with non-existent table/database in impala-shell with the whole server privilege, verified that it results in AnalysisException. - Added CTE test cases for non-existent table/database in AuthorizationStmtTest. - Passed all FE tests. - Passed all exhaustive tests. Change-Id: Ia6b657a7147a136198a9a97a679c9131ee814577 --- M fe/src/main/java/org/apache/impala/analysis/WithClause.java M fe/src/test/java/org/apache/impala/authorization/AuthorizationStmtTest.java 2 files changed, 32 insertions(+), 16 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/23/15123/7 -- To view, visit http://gerrit.cloudera.org:8080/15123 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia6b657a7147a136198a9a97a679c9131ee814577 Gerrit-Change-Number: 15123 Gerrit-PatchSet: 7 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] Reapply IMPALA-9128: part 2: dump traces for slow RPCs
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/15198 ) Change subject: Reapply IMPALA-9128: part 2: dump traces for slow RPCs .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/15198 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic6800b68f61e84420e138bfad67a40be3796b4df Gerrit-Change-Number: 15198 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Mon, 10 Feb 2020 19:23:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4224: execute separate join builds fragments
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14859 ) Change subject: IMPALA-4224: execute separate join builds fragments .. Patch Set 38: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5190/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14859 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4403c8e62d9c13854e7830602ee613f8efc80c58 Gerrit-Change-Number: 14859 Gerrit-PatchSet: 38 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 10 Feb 2020 19:27:51 + Gerrit-HasComments: No
[Impala-ASF-CR] Reapply IMPALA-9128: part 2: dump traces for slow RPCs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15198 ) Change subject: Reapply IMPALA-9128: part 2: dump traces for slow RPCs .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5313/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/15198 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic6800b68f61e84420e138bfad67a40be3796b4df Gerrit-Change-Number: 15198 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Mon, 10 Feb 2020 20:01:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7002: Throw AuthorizationException when user accesss non-existent table/database in CTE without required privileges.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15123 ) Change subject: IMPALA-7002: Throw AuthorizationException when user accesss non-existent table/database in CTE without required privileges. .. Patch Set 7: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5191/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15123 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia6b657a7147a136198a9a97a679c9131ee814577 Gerrit-Change-Number: 15123 Gerrit-PatchSet: 7 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 10 Feb 2020 20:00:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5904: Add full tsan option and fix several TSAN bugs
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15116 ) Change subject: IMPALA-5904: Add full_tsan option and fix several TSAN bugs .. IMPALA-5904: Add full_tsan option and fix several TSAN bugs This patch adds an additional build flag -full_tsan in addition to the existing -tsan build flag. -full_tsan is equivalent to the current -tsan behavior, and -tsan is changed to set the ignore_noninstrumented_modules flag to true. ignore_noninstrumented_modules causes TSAN to ignore any modules that are not TSAN-instrumented. This is necessary to get TSAN to play nicely with Java, since Java is not TSAN-instrumented (see https://wiki.openjdk.java.net/display/tsan/Main and JDK-8208520). While this might decrease the number of issues surfaced by TSAN, it drastically decreases the amount of noise produced by TSAN because the JVM is not running TSAN-instrumented code. Without this flag set to true, almost every single backend test fails with the error: WARNING: ThreadSanitizer: data race (pid=12939) Write of size 1 at 0x7fcbe379c4c6 by thread T31: #0 strncpy /mnt/source/llvm/llvm-5.0.1.src-p2/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:650 (unifiedbetests+0x1b2a4ad) #1 (libjvm.so+0x90e706) This patch fixes various TSAN bugs (e.g. data races) reported while running backend tests and E2E against a TSAN build (it does not make Impala completely TSAN-clean). This patch makes the following changes: * Fixes several bugs involving issues with updating shared variables between threads * Fixes a few race conditions in test classes * Where possible, existing locks are used to fix any data races; in cases where the locking logic is non-trivial, atomics are used * There are a few places where variables are marked as 'volatile' presumably for synchronization purposes; TSAN flags these 'volatile' variables as unsafe, and according to https://github.com/isocpp/CppCoreGuidelines/blob/master/CppCoreGuidelines.md#Rconc-volatile using 'volatile' for synchronization is dangerous; in these cases, the 'volatile' variables are changed to 'atomic' variables * This patch adds a suppression file (bin/tsan-suppresions.txt) similar to the UBSAN suppresion file (bin/ubsan-suppresions.txt) Testing: * Ran exhaustive tests * Ran core tests w/ ASAN build * Manually re-ran backend tests against a TSAN build and made sure the reported errors are gone Change-Id: I3d7ef5c228afd5882e145e6f53885b355d6c25a0 Reviewed-on: http://gerrit.cloudera.org:8080/15116 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M CMakeLists.txt M be/CMakeLists.txt M be/src/common/init.cc M be/src/exec/hdfs-scan-node-base.cc M be/src/exec/hdfs-scan-node-base.h M be/src/exec/hdfs-scan-node-mt.cc M be/src/exec/hdfs-scan-node.cc M be/src/rpc/thrift-thread.cc M be/src/rpc/thrift-thread.h M be/src/runtime/bufferpool/reservation-tracker.cc M be/src/runtime/bufferpool/reservation-tracker.h M be/src/runtime/bufferpool/system-allocator.cc M be/src/runtime/coordinator-backend-state.cc M be/src/runtime/coordinator.cc M be/src/runtime/coordinator.h M be/src/runtime/data-stream-test.cc M be/src/runtime/io/data-cache.cc M be/src/runtime/io/disk-io-mgr-stress.cc M be/src/runtime/io/disk-io-mgr-stress.h M be/src/runtime/io/scan-range.cc M be/src/service/session-expiry-test.cc M be/src/statestore/statestore-test.cc M be/src/util/runtime-profile-test.cc M be/src/util/stopwatch.h M bin/run-backend-tests.sh A bin/tsan-suppressions.txt M buildall.sh M tests/common/environ.py M tests/webserver/test_web_pages.py 29 files changed, 240 insertions(+), 121 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/15116 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I3d7ef5c228afd5882e145e6f53885b355d6c25a0 Gerrit-Change-Number: 15116 Gerrit-PatchSet: 6 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-5904: Add full tsan option and fix several TSAN bugs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15116 ) Change subject: IMPALA-5904: Add full_tsan option and fix several TSAN bugs .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/15116 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3d7ef5c228afd5882e145e6f53885b355d6c25a0 Gerrit-Change-Number: 15116 Gerrit-PatchSet: 5 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 10 Feb 2020 20:49:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9242: Filter privileges before returning them to Sentry
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15068 ) Change subject: IMPALA-9242: Filter privileges before returning them to Sentry .. Patch Set 11: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/15068 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iecd4281368d1c9fe88cfe850ea725cd68895712e Gerrit-Change-Number: 15068 Gerrit-PatchSet: 11 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Anonymous Coward (498) Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Mon, 10 Feb 2020 21:30:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9075: Add support for reading zstd text files
Abhishek Rawat has posted comments on this change. (
http://gerrit.cloudera.org:8080/15023 )
Change subject: IMPALA-9075: Add support for reading zstd text files
..
Patch Set 5:
(2 comments)
Overall looks good. Nice work adding hive interop tests and extending existing
unit tests. I still need to do another round of review. Adding comments so
far...
http://gerrit.cloudera.org:8080/#/c/15023/5/be/src/util/compress.h
File be/src/util/compress.h:
http://gerrit.cloudera.org:8080/#/c/15023/5/be/src/util/compress.h@147
PS5, Line 147: virtual std::string file_extension() const override { return
"zst"; }
I think we should add a comment here explaining that the extension is not used
by parquet. Parquet uses '.parq' as the file extension. Text uses '.zst' as the
extension for files compressed with zstd codec.
http://gerrit.cloudera.org:8080/#/c/15023/5/be/src/util/decompress.cc
File be/src/util/decompress.cc:
http://gerrit.cloudera.org:8080/#/c/15023/5/be/src/util/decompress.cc@616
PS5, Line 616: Status ZstandardDecompressor::Init() {
Init() gets called for both block and streaming code paths. Since it's only
needed for the streaming code path we should only call ZSTD_createDCtx and
ZSTD_freeDCtx in the streaming code path.
--
To view, visit http://gerrit.cloudera.org:8080/15023
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2adce9fe00190558525fa5cd3d50cf5e0f0b0aa4
Gerrit-Change-Number: 15023
Gerrit-PatchSet: 5
Gerrit-Owner: Xiaomeng Zhang
Gerrit-Reviewer: Abhishek Rawat
Gerrit-Reviewer: Andrew Sherman
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Xiaomeng Zhang
Gerrit-Comment-Date: Mon, 10 Feb 2020 22:20:18 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14782 ) Change subject: IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI .. Patch Set 11: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/5311/ -- To view, visit http://gerrit.cloudera.org:8080/14782 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icf74ad06ce1c40fab4ce37de6b7ca78e3e520b43 Gerrit-Change-Number: 14782 Gerrit-PatchSet: 11 Gerrit-Owner: Vincent Tran Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vincent Tran Gerrit-Comment-Date: Mon, 10 Feb 2020 22:21:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8852: Skip short-circuit config check for coordinator-only mode
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15173 ) Change subject: IMPALA-8852: Skip short-circuit config check for coordinator-only mode .. Patch Set 5: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/5312/ -- To view, visit http://gerrit.cloudera.org:8080/15173 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I373d4037f4cee203322a398b77b75810ba708bb5 Gerrit-Change-Number: 15173 Gerrit-PatchSet: 5 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Mon, 10 Feb 2020 22:34:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7002: Throw AuthorizationException when user accesses non-existent table/database in CTE without required privileges.
Wenzhe Zhou has uploaded a new patch set (#8). ( http://gerrit.cloudera.org:8080/15123 ) Change subject: IMPALA-7002: Throw AuthorizationException when user accesses non-existent table/database in CTE without required privileges. .. IMPALA-7002: Throw AuthorizationException when user accesses non-existent table/database in CTE without required privileges. Currently if a user without required privileges tries to access a non-existent database or table, then impala returns an analysis exception instead of authorization exception. This happens because during analysis of the with clause, the authorization request does not get registered due to analysis exception being thrown before it. This patch makes sure that those requests get registered regardless. Testing: - Manual test: - ran CTE with non-existent table/database in impala-shell without privilege, verified that it results in AuthorizationException. - ran CTE with non-existent table/database in impala-shell with the whole server privilege, verified that it results in AnalysisException. - Added CTE test cases for non-existent table/database in AuthorizationStmtTest. - Passed all FE tests. - Passed all exhaustive tests. Change-Id: Ia6b657a7147a136198a9a97a679c9131ee814577 --- M fe/src/main/java/org/apache/impala/analysis/WithClause.java M fe/src/test/java/org/apache/impala/authorization/AuthorizationStmtTest.java 2 files changed, 32 insertions(+), 16 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/23/15123/8 -- To view, visit http://gerrit.cloudera.org:8080/15123 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia6b657a7147a136198a9a97a679c9131ee814577 Gerrit-Change-Number: 15123 Gerrit-PatchSet: 8 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] IMPALA-7002: Throw AuthorizationException when user accesses non-existent table/database in CTE without required privileges.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15123 ) Change subject: IMPALA-7002: Throw AuthorizationException when user accesses non-existent table/database in CTE without required privileges. .. Patch Set 8: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5192/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15123 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia6b657a7147a136198a9a97a679c9131ee814577 Gerrit-Change-Number: 15123 Gerrit-PatchSet: 8 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Tue, 11 Feb 2020 00:38:12 + Gerrit-HasComments: No
[Impala-ASF-CR] Reapply IMPALA-9128: part 2: dump traces for slow RPCs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15198 ) Change subject: Reapply IMPALA-9128: part 2: dump traces for slow RPCs .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/15198 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic6800b68f61e84420e138bfad67a40be3796b4df Gerrit-Change-Number: 15198 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Tue, 11 Feb 2020 00:52:36 + Gerrit-HasComments: No
[Impala-ASF-CR] Reapply IMPALA-9128: part 2: dump traces for slow RPCs
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15198 ) Change subject: Reapply IMPALA-9128: part 2: dump traces for slow RPCs .. Reapply IMPALA-9128: part 2: dump traces for slow RPCs This change was accidentally reverted in the KRPC rebase. It will be upstreamed to Kudu as KUDU-2996. Change-Id: Ic6800b68f61e84420e138bfad67a40be3796b4df Reviewed-on: http://gerrit.cloudera.org:8080/15198 Reviewed-by: Thomas Tauber-Marshall Tested-by: Impala Public Jenkins --- M be/src/kudu/rpc/rpcz_store.cc 1 file changed, 5 insertions(+), 1 deletion(-) Approvals: Thomas Tauber-Marshall: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/15198 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ic6800b68f61e84420e138bfad67a40be3796b4df Gerrit-Change-Number: 15198 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-9340: fix bug where max missed heartbeats is off by one
Riza Suminto has uploaded this change for review. (
http://gerrit.cloudera.org:8080/15201
Change subject: IMPALA-9340: fix bug where max missed heartbeats is off by one
..
IMPALA-9340: fix bug where max missed heartbeats is off by one
Max missed heartbeats is off by one due to greater than sign ('>')
used in comparison in failure-detector.cc. This commit change the sign
to greater than or equal ('>=').
Testing:
* Manual test by running impala mini cluster, kill one of impalad, and
verify in statestored log that the killed impalad is declared as
failed exactly at statestore_max_missed_heartbeats
Change-Id: I19f6bfa7e08d231896665d85299302a17959fb6f
---
M be/src/statestore/failure-detector.cc
1 file changed, 2 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/15201/1
--
To view, visit http://gerrit.cloudera.org:8080/15201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I19f6bfa7e08d231896665d85299302a17959fb6f
Gerrit-Change-Number: 15201
Gerrit-PatchSet: 1
Gerrit-Owner: Riza Suminto
[Impala-ASF-CR] IMPALA-7002: Throw AuthorizationException when user accesses non-existent table/database in CTE without required privileges.
Wenzhe Zhou has uploaded a new patch set (#9). ( http://gerrit.cloudera.org:8080/15123 ) Change subject: IMPALA-7002: Throw AuthorizationException when user accesses non-existent table/database in CTE without required privileges. .. IMPALA-7002: Throw AuthorizationException when user accesses non-existent table/database in CTE without required privileges. Currently if a user without required privileges tries to access a non-existent database or table, then impala returns an analysis exception instead of authorization exception. This happens because during analysis of the with clause, the authorization request does not get registered due to analysis exception being thrown before it. This patch makes sure that those requests get registered regardless. Testing: - Manual test: - ran CTE with non-existent table/database in impala-shell without privilege, verified that it results in AuthorizationException. - ran CTE with non-existent table/database in impala-shell with the whole server privilege, verified that it results in AnalysisException. - Added CTE test cases for non-existent table/database in AuthorizationStmtTest. - Passed all FE tests. - Passed all exhaustive tests. Change-Id: Ia6b657a7147a136198a9a97a679c9131ee814577 --- M fe/src/main/java/org/apache/impala/analysis/WithClause.java M fe/src/test/java/org/apache/impala/authorization/AuthorizationStmtTest.java 2 files changed, 31 insertions(+), 16 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/23/15123/9 -- To view, visit http://gerrit.cloudera.org:8080/15123 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia6b657a7147a136198a9a97a679c9131ee814577 Gerrit-Change-Number: 15123 Gerrit-PatchSet: 9 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] IMPALA-9340: fix bug where max missed heartbeats is off by one
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15201 ) Change subject: IMPALA-9340: fix bug where max missed heartbeats is off by one .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5193/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15201 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I19f6bfa7e08d231896665d85299302a17959fb6f Gerrit-Change-Number: 15201 Gerrit-PatchSet: 1 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 11 Feb 2020 02:21:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7002: Throw AuthorizationException when user accesses non-existent table/database in CTE without required privileges.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15123 ) Change subject: IMPALA-7002: Throw AuthorizationException when user accesses non-existent table/database in CTE without required privileges. .. Patch Set 9: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5194/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15123 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia6b657a7147a136198a9a97a679c9131ee814577 Gerrit-Change-Number: 15123 Gerrit-PatchSet: 9 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Tue, 11 Feb 2020 02:40:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9287: Add support for embedded HMS in CDP builds
wangsheng has uploaded a new patch set (#13). ( http://gerrit.cloudera.org:8080/15057 ) Change subject: IMPALA-9287: Add support for embedded HMS in CDP builds .. IMPALA-9287: Add support for embedded HMS in CDP builds In some situations, an embedded HMS is enough for catalogd server. And we've already implemented this in IMPALA-8974. But after setting USE_CDP_HIVE=true and rebuilt impala, the custom cluster test case test_kudu_table_create_without_hms would failed due to lacking of related jars. The solution is to add related maven dependency in $IMPALA_HOME/fe/pom.xml and $IMPALA_HOME/shaded-deps/pom.xml. Tests: * Ran test_kudu_table_create_without_hms.py by setting USE_CDP_HIVE=true locally Change-Id: Ibc7d7e30cd560d43bb707dec54f4494355809f66 --- M bin/bootstrap_system.sh M fe/pom.xml M fe/src/test/resources/hive-site.xml.py M tests/custom_cluster/test_kudu_table_create_without_hms.py 4 files changed, 36 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/57/15057/13 -- To view, visit http://gerrit.cloudera.org:8080/15057 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ibc7d7e30cd560d43bb707dec54f4494355809f66 Gerrit-Change-Number: 15057 Gerrit-PatchSet: 13 Gerrit-Owner: wangsheng Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: wangsheng
[Impala-ASF-CR] IMPALA-9287: Add support for embedded HMS in CDP builds
wangsheng has posted comments on this change. ( http://gerrit.cloudera.org:8080/15057 ) Change subject: IMPALA-9287: Add support for embedded HMS in CDP builds .. Patch Set 13: (3 comments) > (2 comments) > > > > (1 comment) > > > > The error is: cannot found class PartitionExpressionForMetastore. > I > > add class one by one to keep pom files minimal, all including > four > > classed. > > Ah I see. I think this can be fixed by setting metastore.expression.proxy > to org.apache.hadoop.hive.metastore.DefaultPartitionExpressionProxy > in the hive-site.xml.py if hive_major_version >=3 here: > https://github.com/apache/impala/blob/master/fe/src/test/resources/hive-site.xml.py#L93 > > The PartitionExpressionProxy API is only usable by Hive since it > uses some of its internal classes which are unavailable to non-hive > applications. Refer > https://cwiki.apache.org/confluence/display/Hive/AdminManual+Metastore+3.0+Administration > standalone mode for more details. Thanks for your code view patiently, Vihang. I've already modified code according to your advice. http://gerrit.cloudera.org:8080/#/c/15057/12/fe/pom.xml File fe/pom.xml: http://gerrit.cloudera.org:8080/#/c/15057/12/fe/pom.xml@1065 PS12, Line 1065: > I would avoid making any changes to hive-2 profile since they seem unnecess Done http://gerrit.cloudera.org:8080/#/c/15057/12/fe/pom.xml@1235 PS12, Line 1235: runtime > This dependency can be changed to a runtime scope here for hive-3 profile o Done http://gerrit.cloudera.org:8080/#/c/15057/12/shaded-deps/pom.xml File shaded-deps/pom.xml: http://gerrit.cloudera.org:8080/#/c/15057/12/shaded-deps/pom.xml@99 PS12, Line 99: > See my comment earlier on PartitionExpressionProxy configurations. I think Done -- To view, visit http://gerrit.cloudera.org:8080/15057 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibc7d7e30cd560d43bb707dec54f4494355809f66 Gerrit-Change-Number: 15057 Gerrit-PatchSet: 13 Gerrit-Owner: wangsheng Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: wangsheng Gerrit-Comment-Date: Tue, 11 Feb 2020 02:48:08 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8800: Added support of Kudu DATE type to Impala
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/14705 ) Change subject: IMPALA-8800: Added support of Kudu DATE type to Impala .. Patch Set 13: (3 comments) http://gerrit.cloudera.org:8080/#/c/14705/13/be/src/exec/kudu-util.cc File be/src/exec/kudu-util.cc: http://gerrit.cloudera.org:8080/#/c/14705/13/be/src/exec/kudu-util.cc@133 PS13, Line 133: // If the value was invalid the slot should've been null. What slot and why is it relevant here? http://gerrit.cloudera.org:8080/#/c/14705/13/be/src/exec/kudu-util.cc@135 PS13, Line 135: Invalid DateValue" Any chance to output the value for troubleshooting? http://gerrit.cloudera.org:8080/#/c/14705/13/be/src/exec/kudu-util.cc@195 PS13, Line 195: "Could not set Kudu row value." nit: while you are here, maybe replace this 10+ duplication with a string literal constant and use it in the scope of this method? Feel to ignore if it seems to much of the scope of this patch. -- To view, visit http://gerrit.cloudera.org:8080/14705 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I91656749a58ac769b54c2a63bdd4f85c89520b32 Gerrit-Change-Number: 14705 Gerrit-PatchSet: 13 Gerrit-Owner: Volodymyr Verovkin Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Volodymyr Verovkin Gerrit-Comment-Date: Tue, 11 Feb 2020 02:48:53 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 ) Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/15189/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/15189/1//COMMIT_MSG@20 PS1, Line 20: Tests > Did you think about adding some kind of custom cluster test to prevent bit Good point. Added a test for this. -- To view, visit http://gerrit.cloudera.org:8080/15189 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42 Gerrit-Change-Number: 15189 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Tue, 11 Feb 2020 03:01:18 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster
Hello Fang-Yu Rao, Joe McDonnell, Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/15189 to look at the new patch set (#2). Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster .. IMPALA-9304: Support starting Hive with Ranger in minicluster Add a new flag -with_ranger in testdata/bin/run-hive-server.sh to start Hive with Ranger integration. The relative configuration files are generated in bin/create-test-configuration.sh using a new varient ranger_auth in hive-site.xml.py. Only Hive3 is supported. Current limitation: Can't use different username in Beeline by the -n option. "select current_user()" keeps returning my username, while "select logged_in_user()" can return the username given by -n option but it's not used in authorization. Tests: - Ran bin/create-test-configuration.sh and verified the generated hive-site_ranger_auth.xml contains Ranger configurations. - Ran testdata/bin/run-hive-server.sh -with_ranger. Verified column masking and row filtering policies took effect in Beeline. - Add test in test_ranger.py for this mode. Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42 --- M bin/create-test-configuration.sh M fe/src/test/resources/hive-site.xml.py M testdata/bin/run-hive-server.sh A testdata/workloads/functional-query/queries/QueryTest/hive_ranger_integration.test M tests/authorization/test_ranger.py M tests/common/impala_connection.py M tests/common/skip.py 7 files changed, 93 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/15189/2 -- To view, visit http://gerrit.cloudera.org:8080/15189 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42 Gerrit-Change-Number: 15189 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 ) Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5314/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/15189 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42 Gerrit-Change-Number: 15189 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Tue, 11 Feb 2020 03:04:38 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9287: Add support for embedded HMS in CDP builds
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15057 ) Change subject: IMPALA-9287: Add support for embedded HMS in CDP builds .. Patch Set 13: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5195/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15057 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibc7d7e30cd560d43bb707dec54f4494355809f66 Gerrit-Change-Number: 15057 Gerrit-PatchSet: 13 Gerrit-Owner: wangsheng Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: wangsheng Gerrit-Comment-Date: Tue, 11 Feb 2020 03:32:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 ) Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5196/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15189 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42 Gerrit-Change-Number: 15189 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Tue, 11 Feb 2020 03:47:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9287: Add support for embedded HMS in CDP builds
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/15057 ) Change subject: IMPALA-9287: Add support for embedded HMS in CDP builds .. Patch Set 13: Thanks for making the suggested change. I have triggered a job https://jenkins.impala.io/job/ubuntu-16.04-from-scratch-cdp-hive/62 which will run the core tests with the patch. Lets wait to for it to complete (takes about 3-4 hours). The patch looks good to me. -- To view, visit http://gerrit.cloudera.org:8080/15057 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibc7d7e30cd560d43bb707dec54f4494355809f66 Gerrit-Change-Number: 15057 Gerrit-PatchSet: 13 Gerrit-Owner: wangsheng Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: wangsheng Gerrit-Comment-Date: Tue, 11 Feb 2020 05:02:53 + Gerrit-HasComments: No
[native-toolchain-CR] IMPALA-9279: part 2: Bump Kudu version to 5c610bf40
Attila Jeges has posted comments on this change. ( http://gerrit.cloudera.org:8080/15192 ) Change subject: IMPALA-9279: part 2: Bump Kudu version to 5c610bf40 .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/15192 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3ff92cc5e1de220a4c140cf6c0117b5fa1e89226 Gerrit-Change-Number: 15192 Gerrit-PatchSet: 1 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Tue, 11 Feb 2020 06:40:20 + Gerrit-HasComments: No
[native-toolchain-CR] IMPALA-9279: part 2: Bump Kudu version to 5c610bf40
Attila Jeges has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15192 ) Change subject: IMPALA-9279: part 2: Bump Kudu version to 5c610bf40 .. IMPALA-9279: part 2: Bump Kudu version to 5c610bf40 This pulls in a Kudu change that links Kudu executables statically to libcurl in Kudu's thirdparty directory instead of relying on the dynamic linker to find libcurl at runtime. Testing: - Ran the C6 toolchain build job with the Kudu version bump for native toolchain to make sure that it builds on all supported platforms. Change-Id: I3ff92cc5e1de220a4c140cf6c0117b5fa1e89226 Reviewed-on: http://gerrit.cloudera.org:8080/15192 Reviewed-by: Joe McDonnell Tested-by: Attila Jeges --- M buildall.sh 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Joe McDonnell: Looks good to me, approved Attila Jeges: Verified -- To view, visit http://gerrit.cloudera.org:8080/15192 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I3ff92cc5e1de220a4c140cf6c0117b5fa1e89226 Gerrit-Change-Number: 15192 Gerrit-PatchSet: 2 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Joe McDonnell
[Impala-ASF-CR] IMPALA-8852: Skip short-circuit config check for coordinator-only mode
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15173 ) Change subject: IMPALA-8852: Skip short-circuit config check for coordinator-only mode .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5316/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/15173 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I373d4037f4cee203322a398b77b75810ba708bb5 Gerrit-Change-Number: 15173 Gerrit-PatchSet: 5 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Tue, 11 Feb 2020 07:45:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 ) Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster .. Patch Set 2: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/5314/ -- To view, visit http://gerrit.cloudera.org:8080/15189 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42 Gerrit-Change-Number: 15189 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Tue, 11 Feb 2020 07:57:31 + Gerrit-HasComments: No
