[Impala-ASF-CR] IMPALA-8790: fix referencing wrong grouping exprs of MultiAggregateInfo
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14063 ) Change subject: IMPALA-8790: fix referencing wrong grouping exprs of MultiAggregateInfo .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/14063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I11a80bd6d73ea00ad8c644469558a1885706f596 Gerrit-Change-Number: 14063 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Sat, 17 Aug 2019 05:35:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8790: fix referencing wrong grouping exprs of MultiAggregateInfo
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14063 ) Change subject: IMPALA-8790: fix referencing wrong grouping exprs of MultiAggregateInfo .. IMPALA-8790: fix referencing wrong grouping exprs of MultiAggregateInfo When creating the single node plan for analytic functions (see SingleNodePlanner#createQueryPlan), if the query block contains aggregations, the grouping exprs are carried on for optimizations (see AnalyticPlanner#computeInputPartitionExprs). This patch fixes a regression bug due to IMPALA-110 in this phase. The pre-IMPALA-110 behavior is carrying the groupingExprs of the AggregateInfo (if has). Those exprs are already substituted in AggregationNode#init calling from SingleNodePlanner#createSelectPlan. Now the behavior is carrying the groupingExprs of the MultiAggregateInfo (if has). Those exprs are not substituted in AggregationNode#init. Instead, MultiAggregateInfo creates a substituted grouping exprs in this step. They are what we actually need. The original grouping exprs may use non-materialized slots, which should not be referenced in exchanges. Also add some useful TRACE logs for future debugging. Tests - Add planner tests to cover the regression bug. - Run CORE tests Change-Id: I11a80bd6d73ea00ad8c644469558a1885706f596 Reviewed-on: http://gerrit.cloudera.org:8080/14063 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M fe/src/main/java/org/apache/impala/planner/AnalyticPlanner.java M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M testdata/workloads/functional-planner/queries/PlannerTest/analytic-fns.test 4 files changed, 109 insertions(+), 3 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/14063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I11a80bd6d73ea00ad8c644469558a1885706f596 Gerrit-Change-Number: 14063 Gerrit-PatchSet: 4 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-8872: Override httpcore from libthrift to 4.4.9
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14087 ) Change subject: IMPALA-8872: Override httpcore from libthrift to 4.4.9 .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4295/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14087 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I675356d002354b0aff439f0635e30f4610a96989 Gerrit-Change-Number: 14087 Gerrit-PatchSet: 3 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Sat, 17 Aug 2019 05:02:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8872: Override httpcore from libthrift to 4.4.9
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/14087 ) Change subject: IMPALA-8872: Override httpcore from libthrift to 4.4.9 .. Patch Set 3: Code-Review+2 (2 comments) thanks for the quick review. Carrying +2. (Not rerunning the GVO because I just fixed comments, will carry it once the job finishes). http://gerrit.cloudera.org:8080/#/c/14087/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/14087/2//COMMIT_MSG@7 PS2, Line 7: 4.4.9 > Should this say "4.4.9"? Done http://gerrit.cloudera.org:8080/#/c/14087/2/fe/pom.xml File fe/pom.xml: http://gerrit.cloudera.org:8080/#/c/14087/2/fe/pom.xml@361 PS2, Line 361: to ke > nit: double word Done -- To view, visit http://gerrit.cloudera.org:8080/14087 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I675356d002354b0aff439f0635e30f4610a96989 Gerrit-Change-Number: 14087 Gerrit-PatchSet: 3 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Sat, 17 Aug 2019 04:23:25 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8872: Override httpcore from libthrift to 4.4.9
Hello Lars Volker, Vihang Karajgaonkar, Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14087 to look at the new patch set (#3). Change subject: IMPALA-8872: Override httpcore from libthrift to 4.4.9 .. IMPALA-8872: Override httpcore from libthrift to 4.4.9 When deploying a cluster with Ranger publishing audits to SOLR, we noticed that the SPNEGO request consistently fails with httpcore 4.4.1 pulled in by thrift. The error goes away with httpcore v4.4.9 (manually tested). This patch overrides the dependency. Change-Id: I675356d002354b0aff439f0635e30f4610a96989 --- M fe/pom.xml 1 file changed, 16 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/87/14087/3 -- To view, visit http://gerrit.cloudera.org:8080/14087 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I675356d002354b0aff439f0635e30f4610a96989 Gerrit-Change-Number: 14087 Gerrit-PatchSet: 3 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8868: Fix 401 response when LDAP and Kerberos are enabled
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14077 ) Change subject: IMPALA-8868: Fix 401 response when LDAP and Kerberos are enabled .. IMPALA-8868: Fix 401 response when LDAP and Kerberos are enabled When both kerberos and ldap auth are enabled and an http request is not successfully authenticated, THttpServer only sends the 'WWW-Authenticate: Basic' challenge and doesn't send the 'WWW-Authenticate: Negotiate' challenge, which can cause clients that want to connect with kerberos to fail to authenticate. This patch fixes this to send both challenges. Testing: - Manually tested in a cluster with both Kerberos and LDAP enabled on Impala with connections proxied through Apache Knox, which would previously fail. Change-Id: I138f33783bfd0f8f9b8db242589a9cc75cfd392a Reviewed-on: http://gerrit.cloudera.org:8080/14077 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/transport/THttpServer.cpp 1 file changed, 10 insertions(+), 5 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/14077 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I138f33783bfd0f8f9b8db242589a9cc75cfd392a Gerrit-Change-Number: 14077 Gerrit-PatchSet: 4 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-8868: Fix 401 response when LDAP and Kerberos are enabled
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14077 ) Change subject: IMPALA-8868: Fix 401 response when LDAP and Kerberos are enabled .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/14077 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I138f33783bfd0f8f9b8db242589a9cc75cfd392a Gerrit-Change-Number: 14077 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Sat, 17 Aug 2019 04:10:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8872: Override httpcore from libthrift to 4.4.6
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/14087 ) Change subject: IMPALA-8872: Override httpcore from libthrift to 4.4.6 .. Patch Set 2: Code-Review+2 (2 comments) Thanks for fixing this. LGTM but had 2 nits. http://gerrit.cloudera.org:8080/#/c/14087/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/14087/2//COMMIT_MSG@7 PS2, Line 7: 4.4.6 Should this say "4.4.9"? http://gerrit.cloudera.org:8080/#/c/14087/2/fe/pom.xml File fe/pom.xml: http://gerrit.cloudera.org:8080/#/c/14087/2/fe/pom.xml@361 PS2, Line 361: to to nit: double word -- To view, visit http://gerrit.cloudera.org:8080/14087 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I675356d002354b0aff439f0635e30f4610a96989 Gerrit-Change-Number: 14087 Gerrit-PatchSet: 2 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Sat, 17 Aug 2019 04:04:34 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8872: Override httpcore from libthrift to 4.4.6
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14087 ) Change subject: IMPALA-8872: Override httpcore from libthrift to 4.4.6 .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4294/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14087 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I675356d002354b0aff439f0635e30f4610a96989 Gerrit-Change-Number: 14087 Gerrit-PatchSet: 2 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Sat, 17 Aug 2019 03:15:46 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8872: Override httpcore from libthrift to 4.4.6
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14087 ) Change subject: IMPALA-8872: Override httpcore from libthrift to 4.4.6 .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4293/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14087 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I675356d002354b0aff439f0635e30f4610a96989 Gerrit-Change-Number: 14087 Gerrit-PatchSet: 1 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Sat, 17 Aug 2019 03:07:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7935: Disable /catalog object in local catalog mode.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12443 ) Change subject: IMPALA-7935: Disable /catalog_object in local catalog mode. .. Patch Set 10: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12443 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia04797b32964c2edaa2e860dcf510d6f9cccd81c Gerrit-Change-Number: 12443 Gerrit-PatchSet: 10 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Sat, 17 Aug 2019 02:44:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7935: Disable /catalog object in local catalog mode.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12443 ) Change subject: IMPALA-7935: Disable /catalog_object in local catalog mode. .. Patch Set 10: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4815/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/12443 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia04797b32964c2edaa2e860dcf510d6f9cccd81c Gerrit-Change-Number: 12443 Gerrit-PatchSet: 10 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Sat, 17 Aug 2019 02:44:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7935: Disable /catalog object in local catalog mode.
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/12443 ) Change subject: IMPALA-7935: Disable /catalog_object in local catalog mode. .. Patch Set 9: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12443 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia04797b32964c2edaa2e860dcf510d6f9cccd81c Gerrit-Change-Number: 12443 Gerrit-PatchSet: 9 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Sat, 17 Aug 2019 02:44:16 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8872: Override httpcore from libthrift to 4.4.6
Hello Vihang Karajgaonkar, Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14087 to look at the new patch set (#2). Change subject: IMPALA-8872: Override httpcore from libthrift to 4.4.6 .. IMPALA-8872: Override httpcore from libthrift to 4.4.6 When deploying a cluster with Ranger publishing audits to SOLR, we noticed that the SPNEGO request consistently fails with httpcore 4.4.1 pulled in by thrift. The error goes away with httpcore v4.4.9 (manually tested). This patch overrides the dependency. Change-Id: I675356d002354b0aff439f0635e30f4610a96989 --- M fe/pom.xml 1 file changed, 16 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/87/14087/2 -- To view, visit http://gerrit.cloudera.org:8080/14087 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I675356d002354b0aff439f0635e30f4610a96989 Gerrit-Change-Number: 14087 Gerrit-PatchSet: 2 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8872: Override httpcore from libthrift to 4.4.6
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14087 ) Change subject: IMPALA-8872: Override httpcore from libthrift to 4.4.6 .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4814/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/14087 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I675356d002354b0aff439f0635e30f4610a96989 Gerrit-Change-Number: 14087 Gerrit-PatchSet: 2 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Sat, 17 Aug 2019 02:36:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8872: Override httpcore from libthrift to 4.4.6
Bharath Vissapragada has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14087 Change subject: IMPALA-8872: Override httpcore from libthrift to 4.4.6 .. IMPALA-8872: Override httpcore from libthrift to 4.4.6 When deploying a cluster with Ranger publishing audits to SOLR, we noticed that the SPNEGO request consistently fails with httpcore 4.4.1 pulled in by thrift. The error goes away with httpcore v4.4.6 (manually tested). This patch overrides the dependency. Change-Id: I675356d002354b0aff439f0635e30f4610a96989 --- M fe/pom.xml 1 file changed, 9 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/87/14087/1 -- To view, visit http://gerrit.cloudera.org:8080/14087 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I675356d002354b0aff439f0635e30f4610a96989 Gerrit-Change-Number: 14087 Gerrit-PatchSet: 1 Gerrit-Owner: Bharath Vissapragada
[Impala-ASF-CR] IMPALA-8474: [DOCS] Impala 3.3 Release Notes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14054 ) Change subject: IMPALA-8474: [DOCS] Impala 3.3 Release Notes .. Patch Set 4: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/458/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/14054 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I02bb1944c57866b797b1856af139800445e4ac18 Gerrit-Change-Number: 14054 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Sat, 17 Aug 2019 01:44:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8790: fix referencing wrong grouping exprs of MultiAggregateInfo
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14063 ) Change subject: IMPALA-8790: fix referencing wrong grouping exprs of MultiAggregateInfo .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4813/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/14063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I11a80bd6d73ea00ad8c644469558a1885706f596 Gerrit-Change-Number: 14063 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Sat, 17 Aug 2019 01:30:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8790: fix referencing wrong grouping exprs of MultiAggregateInfo
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14063 ) Change subject: IMPALA-8790: fix referencing wrong grouping exprs of MultiAggregateInfo .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I11a80bd6d73ea00ad8c644469558a1885706f596 Gerrit-Change-Number: 14063 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Sat, 17 Aug 2019 01:30:02 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8474: [DOCS] Impala 3.3 Release Notes
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14054 to look at the new patch set (#4). Change subject: IMPALA-8474: [DOCS] Impala 3.3 Release Notes .. IMPALA-8474: [DOCS] Impala 3.3 Release Notes Change-Id: I02bb1944c57866b797b1856af139800445e4ac18 --- M docs/impala_keydefs.ditamap M docs/topics/impala_incompatible_changes.xml M docs/topics/impala_new_features.xml 3 files changed, 842 insertions(+), 951 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/54/14054/4 -- To view, visit http://gerrit.cloudera.org:8080/14054 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I02bb1944c57866b797b1856af139800445e4ac18 Gerrit-Change-Number: 14054 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8474: [DOCS] Impala 3.3 Release Notes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14054 ) Change subject: IMPALA-8474: [DOCS] Impala 3.3 Release Notes .. Patch Set 4: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/458/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/14054 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I02bb1944c57866b797b1856af139800445e4ac18 Gerrit-Change-Number: 14054 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Sat, 17 Aug 2019 01:18:56 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8871: Upgrade Thrift version in fe
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14084 ) Change subject: IMPALA-8871: Upgrade Thrift version in fe .. IMPALA-8871: Upgrade Thrift version in fe Upgrade fe/ Thrift version to 0.9.3-1. Testing: * Ran exhaustive tests. * Ran perf-AB-test, which showed no significant perf difference. Change-Id: If62e6330f0a725ebd4213c200656fc12a0eccf06 Reviewed-on: http://gerrit.cloudera.org:8080/14084 Reviewed-by: Tim Armstrong Tested-by: Impala Public Jenkins --- M impala-parent/pom.xml 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Tim Armstrong: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/14084 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: If62e6330f0a725ebd4213c200656fc12a0eccf06 Gerrit-Change-Number: 14084 Gerrit-PatchSet: 2 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8871: Upgrade Thrift version in fe
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14084 ) Change subject: IMPALA-8871: Upgrade Thrift version in fe .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/14084 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If62e6330f0a725ebd4213c200656fc12a0eccf06 Gerrit-Change-Number: 14084 Gerrit-PatchSet: 1 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Sat, 17 Aug 2019 00:57:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8868: Fix 401 response when LDAP and Kerberos are enabled
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14077 ) Change subject: IMPALA-8868: Fix 401 response when LDAP and Kerberos are enabled .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4292/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14077 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I138f33783bfd0f8f9b8db242589a9cc75cfd392a Gerrit-Change-Number: 14077 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Sat, 17 Aug 2019 00:43:45 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8868: Fix 401 response when LDAP and Kerberos are enabled
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14077 ) Change subject: IMPALA-8868: Fix 401 response when LDAP and Kerberos are enabled .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14077 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I138f33783bfd0f8f9b8db242589a9cc75cfd392a Gerrit-Change-Number: 14077 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Sat, 17 Aug 2019 00:05:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8868: Fix 401 response when LDAP and Kerberos are enabled
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14077 ) Change subject: IMPALA-8868: Fix 401 response when LDAP and Kerberos are enabled .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4812/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/14077 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I138f33783bfd0f8f9b8db242589a9cc75cfd392a Gerrit-Change-Number: 14077 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Sat, 17 Aug 2019 00:05:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8868: Fix 401 response when LDAP and Kerberos are enabled
Hello Todd Lipcon, Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14077 to look at the new patch set (#2). Change subject: IMPALA-8868: Fix 401 response when LDAP and Kerberos are enabled .. IMPALA-8868: Fix 401 response when LDAP and Kerberos are enabled When both kerberos and ldap auth are enabled and an http request is not successfully authenticated, THttpServer only sends the 'WWW-Authenticate: Basic' challenge and doesn't send the 'WWW-Authenticate: Negotiate' challenge, which can cause clients that want to connect with kerberos to fail to authenticate. This patch fixes this to send both challenges. Testing: - Manually tested in a cluster with both Kerberos and LDAP enabled on Impala with connections proxied through Apache Knox, which would previously fail. Change-Id: I138f33783bfd0f8f9b8db242589a9cc75cfd392a --- M be/src/transport/THttpServer.cpp 1 file changed, 10 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/77/14077/2 -- To view, visit http://gerrit.cloudera.org:8080/14077 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I138f33783bfd0f8f9b8db242589a9cc75cfd392a Gerrit-Change-Number: 14077 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-8868: Fix 401 response when LDAP and Kerberos are enabled
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/14077 ) Change subject: IMPALA-8868: Fix 401 response when LDAP and Kerberos are enabled .. Patch Set 2: Code-Review+2 (1 comment) carrying forward http://gerrit.cloudera.org:8080/#/c/14077/1/be/src/transport/THttpServer.cpp File be/src/transport/THttpServer.cpp: http://gerrit.cloudera.org:8080/#/c/14077/1/be/src/transport/THttpServer.cpp@175 PS1, Line 175: TryStripPrefixString(auth_value_, "Basic ", _basic_auth_token); > The API of TryStripPrefixString() is weird. I find the flow a little confus Done -- To view, visit http://gerrit.cloudera.org:8080/14077 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I138f33783bfd0f8f9b8db242589a9cc75cfd392a Gerrit-Change-Number: 14077 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Sat, 17 Aug 2019 00:04:39 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8868: Fix 401 response when LDAP and Kerberos are enabled
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14077 ) Change subject: IMPALA-8868: Fix 401 response when LDAP and Kerberos are enabled .. Patch Set 1: (That comment isn't blocking) -- To view, visit http://gerrit.cloudera.org:8080/14077 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I138f33783bfd0f8f9b8db242589a9cc75cfd392a Gerrit-Change-Number: 14077 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 16 Aug 2019 23:37:35 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8868: Fix 401 response when LDAP and Kerberos are enabled
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14077 ) Change subject: IMPALA-8868: Fix 401 response when LDAP and Kerberos are enabled .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14077 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I138f33783bfd0f8f9b8db242589a9cc75cfd392a Gerrit-Change-Number: 14077 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 16 Aug 2019 23:37:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8868: Fix 401 response when LDAP and Kerberos are enabled
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14077 ) Change subject: IMPALA-8868: Fix 401 response when LDAP and Kerberos are enabled .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/14077/1/be/src/transport/THttpServer.cpp File be/src/transport/THttpServer.cpp: http://gerrit.cloudera.org:8080/#/c/14077/1/be/src/transport/THttpServer.cpp@175 PS1, Line 175: if (!got_basic_auth) basic_auth_token = ""; The API of TryStripPrefixString() is weird. I find the flow a little confusing with how it's mutating basic_auth_token then undoing it. Maybe it would be more obvious if you used a temporary for the output argument of TryStripPrefixString(). I.e. something like string stripped_basic_auth_token; bool got_basic_auth = TryStripPrefixString(auth_value_, "Basic ", _auth_token) string basic_auth_token = got_basic_auth ? move(stripped_basic_auth_token) : ""; Same for negotiate_auth_token below. -- To view, visit http://gerrit.cloudera.org:8080/14077 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I138f33783bfd0f8f9b8db242589a9cc75cfd392a Gerrit-Change-Number: 14077 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 16 Aug 2019 23:35:31 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8836: Support COMPUTE STATS on insert only ACID tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14066 ) Change subject: IMPALA-8836: Support COMPUTE STATS on insert only ACID tables .. Patch Set 10: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4291/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14066 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5c06b4678c1ff75c5aa1586a78afea563e64057f Gerrit-Change-Number: 14066 Gerrit-PatchSet: 10 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 16 Aug 2019 23:35:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8836: Support COMPUTE STATS on insert only ACID tables
Tim Armstrong has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14066 ) Change subject: IMPALA-8836: Support COMPUTE STATS on insert only ACID tables .. IMPALA-8836: Support COMPUTE STATS on insert only ACID tables For ACID tables COMPUTE STATS needs to use a new HMS API, as the old one is rejected by metastore. This API currently has some counter intuitive parts: - setPartitionColumnStatistics is used to set table stats, as there is no similar function exposed by HMS client for tables at the moment. - A new writeId is allocated for the stat change, and this needs a transaction, so a transaction is opened/committed/aborted even though this doesn't seem necessary. The Hive code seems to use internal API for this. - Even though the HMS thrift Table object has a colStats field, it is only applied during alter_table if there are other changes like new columns in the tables, so alter_table couldn't be used to change column stats. Additional changes: - DROP STATS is no longer allowed for transactional tables, as it turned out that there is no transactional version of the old API. - Remove COLUMN_STATS_ACCURATE table property during COMPUTE STATS to ensure that Hive does use stats computed by Impala to return answer queries like SELECT count(*) - Changed CatalogOpExecutor.updateCatalog() to get the writeIds earlier. This can mean unnecassary HMS RPC calls if no property change is needed in the end, but I felt it hard to reason about what happens if these RPC calls fail at their original location. TODOs (My plan is to do these in IMPALA-8865): - Tried to make the MetastoreShim API easier to use by adding a class to encapsulate thing like txnId and writeId, but it feels rather half baked and under documented. A similar class is added in https://gerrit.cloudera.org/#/c/14071/, it would be good to merge them. - The validWriteIdList of the original SELECT(s) behind COMPUTE STATS could be used in the HMS API calls, but this would need more plumbing. Change-Id: I5c06b4678c1ff75c5aa1586a78afea563e64057f Reviewed-on: http://gerrit.cloudera.org:8080/14066 Reviewed-by: Tim Armstrong Tested-by: Tim Armstrong --- M fe/src/compat-hive-2/java/org/apache/impala/compat/MetastoreShim.java M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java M fe/src/main/java/org/apache/impala/analysis/DropStatsStmt.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/test/java/org/apache/impala/analysis/AnalyzerTest.java A testdata/workloads/functional-query/queries/QueryTest/acid-compute-stats.test M testdata/workloads/functional-query/queries/QueryTest/acid-negative.test M tests/query_test/test_acid.py 10 files changed, 406 insertions(+), 119 deletions(-) Approvals: Tim Armstrong: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/14066 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I5c06b4678c1ff75c5aa1586a78afea563e64057f Gerrit-Change-Number: 14066 Gerrit-PatchSet: 11 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-8836: Support COMPUTE STATS on insert only ACID tables
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14066 ) Change subject: IMPALA-8836: Support COMPUTE STATS on insert only ACID tables .. Patch Set 10: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/14066 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5c06b4678c1ff75c5aa1586a78afea563e64057f Gerrit-Change-Number: 14066 Gerrit-PatchSet: 10 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 16 Aug 2019 23:22:50 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8124: Modify TestWebPage::test catalog to avoid flakiness.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14075 ) Change subject: IMPALA-8124: Modify TestWebPage::test_catalog to avoid flakiness. .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/14075 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I341bf25baf8d9316a21a9eff860de84b33afd12f Gerrit-Change-Number: 14075 Gerrit-PatchSet: 2 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 16 Aug 2019 23:19:42 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8124: Modify TestWebPage::test catalog to avoid flakiness.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14075 ) Change subject: IMPALA-8124: Modify TestWebPage::test_catalog to avoid flakiness. .. IMPALA-8124: Modify TestWebPage::test_catalog to avoid flakiness. This test scrapes the /catalog webpage for metrics. This can occasionally run into a race condition when a table lock is being held for functional*.* tables and the test tries to get the metrics. This result in a failed assert for the metrics. This change rewrites the test to create new tables with unique_database fixture to avoid flakiness. Also, seperated out an assert with AND into two different asserts so it's easy to track which one failed. Testing: Ran multiple runs of this test locally while refreshing the table. Change-Id: I341bf25baf8d9316a21a9eff860de84b33afd12f Reviewed-on: http://gerrit.cloudera.org:8080/14075 Reviewed-by: Bharath Vissapragada Tested-by: Impala Public Jenkins --- M tests/webserver/test_web_pages.py 1 file changed, 41 insertions(+), 14 deletions(-) Approvals: Bharath Vissapragada: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/14075 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I341bf25baf8d9316a21a9eff860de84b33afd12f Gerrit-Change-Number: 14075 Gerrit-PatchSet: 3 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8841: Try to fix Tez related dataload flakiness
Tim Armstrong has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14081 ) Change subject: IMPALA-8841: Try to fix Tez related dataload flakiness .. IMPALA-8841: Try to fix Tez related dataload flakiness The flakiness may be related to starting Hive queries in parallel which triggers initializing Tez resources in parallel (only needed at the first statement that uses Tez). Doing a non-parallel statement at first may solve the issue. Also includes a fix for a recent issue in 'build-and-copy-hive-udfs' introduced by the version bump in https://gerrit.cloudera.org/#/c/14043/ Change-Id: Id21d57483fe7a4f72f450fb71f8f53b3c1ef6327 Reviewed-on: http://gerrit.cloudera.org:8080/14081 Reviewed-by: Vihang Karajgaonkar Reviewed-by: Tim Armstrong Tested-by: Tim Armstrong --- M testdata/bin/create-load-data.sh M tests/test-hive-udfs/pom.xml 2 files changed, 19 insertions(+), 0 deletions(-) Approvals: Vihang Karajgaonkar: Looks good to me, approved Tim Armstrong: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/14081 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Id21d57483fe7a4f72f450fb71f8f53b3c1ef6327 Gerrit-Change-Number: 14081 Gerrit-PatchSet: 4 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Anonymous Coward (521) Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8841: Try to fix Tez related dataload flakiness
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14081 ) Change subject: IMPALA-8841: Try to fix Tez related dataload flakiness .. Patch Set 3: Verified+1 Code-Review+2 Change from 2->3 was only a comment, I'll just carry the verification. -- To view, visit http://gerrit.cloudera.org:8080/14081 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id21d57483fe7a4f72f450fb71f8f53b3c1ef6327 Gerrit-Change-Number: 14081 Gerrit-PatchSet: 3 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Anonymous Coward (521) Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Fri, 16 Aug 2019 22:59:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8836: Support COMPUTE STATS on insert only ACID tables
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14066 ) Change subject: IMPALA-8836: Support COMPUTE STATS on insert only ACID tables .. Patch Set 10: Code-Review+2 Test just should've been skipped on Hive 2. I'll run it manually on Hive 3 just to check it works. -- To view, visit http://gerrit.cloudera.org:8080/14066 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5c06b4678c1ff75c5aa1586a78afea563e64057f Gerrit-Change-Number: 14066 Gerrit-PatchSet: 10 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 16 Aug 2019 22:55:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8836: Support COMPUTE STATS on insert only ACID tables
Tim Armstrong has uploaded a new patch set (#10) to the change originally created by Csaba Ringhofer. ( http://gerrit.cloudera.org:8080/14066 ) Change subject: IMPALA-8836: Support COMPUTE STATS on insert only ACID tables .. IMPALA-8836: Support COMPUTE STATS on insert only ACID tables For ACID tables COMPUTE STATS needs to use a new HMS API, as the old one is rejected by metastore. This API currently has some counter intuitive parts: - setPartitionColumnStatistics is used to set table stats, as there is no similar function exposed by HMS client for tables at the moment. - A new writeId is allocated for the stat change, and this needs a transaction, so a transaction is opened/committed/aborted even though this doesn't seem necessary. The Hive code seems to use internal API for this. - Even though the HMS thrift Table object has a colStats field, it is only applied during alter_table if there are other changes like new columns in the tables, so alter_table couldn't be used to change column stats. Additional changes: - DROP STATS is no longer allowed for transactional tables, as it turned out that there is no transactional version of the old API. - Remove COLUMN_STATS_ACCURATE table property during COMPUTE STATS to ensure that Hive does use stats computed by Impala to return answer queries like SELECT count(*) - Changed CatalogOpExecutor.updateCatalog() to get the writeIds earlier. This can mean unnecassary HMS RPC calls if no property change is needed in the end, but I felt it hard to reason about what happens if these RPC calls fail at their original location. TODOs (My plan is to do these in IMPALA-8865): - Tried to make the MetastoreShim API easier to use by adding a class to encapsulate thing like txnId and writeId, but it feels rather half baked and under documented. A similar class is added in https://gerrit.cloudera.org/#/c/14071/, it would be good to merge them. - The validWriteIdList of the original SELECT(s) behind COMPUTE STATS could be used in the HMS API calls, but this would need more plumbing. Change-Id: I5c06b4678c1ff75c5aa1586a78afea563e64057f --- M fe/src/compat-hive-2/java/org/apache/impala/compat/MetastoreShim.java M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java M fe/src/main/java/org/apache/impala/analysis/DropStatsStmt.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/test/java/org/apache/impala/analysis/AnalyzerTest.java A testdata/workloads/functional-query/queries/QueryTest/acid-compute-stats.test M testdata/workloads/functional-query/queries/QueryTest/acid-negative.test M tests/query_test/test_acid.py 10 files changed, 406 insertions(+), 119 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/66/14066/10 -- To view, visit http://gerrit.cloudera.org:8080/14066 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5c06b4678c1ff75c5aa1586a78afea563e64057f Gerrit-Change-Number: 14066 Gerrit-PatchSet: 10 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-8866: Bump versions of dependencies
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14083 ) Change subject: IMPALA-8866: Bump versions of dependencies .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/14083 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia60d1ffec034790205404550c4bac54b247547d7 Gerrit-Change-Number: 14083 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 16 Aug 2019 22:47:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8847: Fix test event processing.py on Hive-3
Vihang Karajgaonkar has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14082 ) Change subject: IMPALA-8847: Fix test_event_processing.py on Hive-3 .. IMPALA-8847: Fix test_event_processing.py on Hive-3 The test_event_processing fails on Hive-3 due to a error in the syntax of the hive query for transactional tables. It was missed unfortunately when IMPALA-8847 was merged. Testing done: Ran the test on both Hive-2 and Hive-3 and it passes now Change-Id: I6507540bfbc0d131a061865ed9ed94792ccfa758 Reviewed-on: http://gerrit.cloudera.org:8080/14082 Tested-by: Impala Public Jenkins Reviewed-by: Vihang Karajgaonkar --- M tests/custom_cluster/test_event_processing.py 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Impala Public Jenkins: Verified Vihang Karajgaonkar: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/14082 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I6507540bfbc0d131a061865ed9ed94792ccfa758 Gerrit-Change-Number: 14082 Gerrit-PatchSet: 3 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8847: Fix test event processing.py on Hive-3
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14082 ) Change subject: IMPALA-8847: Fix test_event_processing.py on Hive-3 .. Patch Set 2: Code-Review+2 Carrying forward Csaba's +2 from before -- To view, visit http://gerrit.cloudera.org:8080/14082 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6507540bfbc0d131a061865ed9ed94792ccfa758 Gerrit-Change-Number: 14082 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Fri, 16 Aug 2019 22:16:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8661 : Add randomized tests to stress MetastoreEventsProcessor
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/13932 ) Change subject: IMPALA-8661 : Add randomized tests to stress MetastoreEventsProcessor .. Patch Set 9: Code-Review+2 carrying forward Bharath's +2 from earlier. -- To view, visit http://gerrit.cloudera.org:8080/13932 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8c85b83efd4f56b5ae0e8d1dc6a2ee2feb6721ce Gerrit-Change-Number: 13932 Gerrit-PatchSet: 9 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Fri, 16 Aug 2019 22:14:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8661 : Add randomized tests to stress MetastoreEventsProcessor
Vihang Karajgaonkar has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13932 ) Change subject: IMPALA-8661 : Add randomized tests to stress MetastoreEventsProcessor .. IMPALA-8661 : Add randomized tests to stress MetastoreEventsProcessor This change adds a new stress test for MetastoreEventsProcessor. This test randomly executes hive queries to generate a lot of events. The event processor is invoked at random intervals so that a variable batch of events is processed everytime. After each batch is processed, the test checks the status of events processor. By default, on CDH builds the test is configured to run with 4 concurrent Hive clients and each of the client runs 50 random Hive queries. These defaults can be overridden by passing system properties using maven command arguments "-DnumClients" and "-DnumQueriesPerClients". Additionally, the test also creates impala clients which keep issuing refresh table commands on the test databases to make sure that eventProcessor is doing some real work rather than invalidating/refreshing tables which are already incomplete. This test is added as a junit test and uses Hive JDBC to issue the sqls. This is much faster than the end-to-end python test which issues each hive query in a separate beeline sessions which re-establishes the connection every time. Notes: 1. Ran the test with defaults. It generates about 500 events and runs for close to 4.5 min. This can be changed to a lower value if we see significant increased delay in the test job runtimes. 3. On CDP builds the concurrent hive queries run very slow due to container provisioning time on the minicluster. I have left this as a TODO to investigate. The test runs in single threaded mode with increased number of queries when running against Hive-3 Change-Id: I8c85b83efd4f56b5ae0e8d1dc6a2ee2feb6721ce Reviewed-on: http://gerrit.cloudera.org:8080/13932 Tested-by: Impala Public Jenkins Reviewed-by: Vihang Karajgaonkar --- M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java A fe/src/test/java/org/apache/impala/catalog/events/EventsProcessorStressTest.java A fe/src/test/java/org/apache/impala/testutil/HiveJdbcClientPool.java M fe/src/test/java/org/apache/impala/testutil/ImpalaJdbcClient.java M fe/src/test/java/org/apache/impala/testutil/TestUtils.java A fe/src/test/java/org/apache/impala/util/RandomHiveQueryRunner.java 6 files changed, 1,598 insertions(+), 24 deletions(-) Approvals: Impala Public Jenkins: Verified Vihang Karajgaonkar: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/13932 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I8c85b83efd4f56b5ae0e8d1dc6a2ee2feb6721ce Gerrit-Change-Number: 13932 Gerrit-PatchSet: 10 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8847: Fix test event processing.py on Hive-3
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14082 ) Change subject: IMPALA-8847: Fix test_event_processing.py on Hive-3 .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/14082 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6507540bfbc0d131a061865ed9ed94792ccfa758 Gerrit-Change-Number: 14082 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 16 Aug 2019 22:06:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8836: Support COMPUTE STATS on insert only ACID tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14066 ) Change subject: IMPALA-8836: Support COMPUTE STATS on insert only ACID tables .. Patch Set 9: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4807/ -- To view, visit http://gerrit.cloudera.org:8080/14066 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5c06b4678c1ff75c5aa1586a78afea563e64057f Gerrit-Change-Number: 14066 Gerrit-PatchSet: 9 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 16 Aug 2019 21:12:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8871: Upgrade Thrift version in fe
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14084 ) Change subject: IMPALA-8871: Upgrade Thrift version in fe .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4811/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/14084 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If62e6330f0a725ebd4213c200656fc12a0eccf06 Gerrit-Change-Number: 14084 Gerrit-PatchSet: 1 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 16 Aug 2019 20:47:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7935: Disable /catalog object in local catalog mode.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12443 ) Change subject: IMPALA-7935: Disable /catalog_object in local catalog mode. .. Patch Set 9: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4290/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12443 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia04797b32964c2edaa2e860dcf510d6f9cccd81c Gerrit-Change-Number: 12443 Gerrit-PatchSet: 9 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 16 Aug 2019 20:41:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8871: Upgrade Thrift version in fe
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14084 ) Change subject: IMPALA-8871: Upgrade Thrift version in fe .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4289/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14084 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If62e6330f0a725ebd4213c200656fc12a0eccf06 Gerrit-Change-Number: 14084 Gerrit-PatchSet: 1 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 16 Aug 2019 20:35:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8661 : Add randomized tests to stress MetastoreEventsProcessor
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13932 ) Change subject: IMPALA-8661 : Add randomized tests to stress MetastoreEventsProcessor .. Patch Set 9: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13932 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8c85b83efd4f56b5ae0e8d1dc6a2ee2feb6721ce Gerrit-Change-Number: 13932 Gerrit-PatchSet: 9 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Fri, 16 Aug 2019 20:33:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8841: Try to fix Tez related dataload flakiness
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14081 ) Change subject: IMPALA-8841: Try to fix Tez related dataload flakiness .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/14081 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id21d57483fe7a4f72f450fb71f8f53b3c1ef6327 Gerrit-Change-Number: 14081 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Anonymous Coward (521) Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Fri, 16 Aug 2019 20:30:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8866: Bump versions of dependencies
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14083 ) Change subject: IMPALA-8866: Bump versions of dependencies .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4288/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14083 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia60d1ffec034790205404550c4bac54b247547d7 Gerrit-Change-Number: 14083 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 16 Aug 2019 20:27:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7935: Disable /catalog object in local catalog mode.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12443 ) Change subject: IMPALA-7935: Disable /catalog_object in local catalog mode. .. Patch Set 8: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4287/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12443 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia04797b32964c2edaa2e860dcf510d6f9cccd81c Gerrit-Change-Number: 12443 Gerrit-PatchSet: 8 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 16 Aug 2019 19:54:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8841: Try to fix Tez related dataload flakiness
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14081 ) Change subject: IMPALA-8841: Try to fix Tez related dataload flakiness .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4286/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14081 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id21d57483fe7a4f72f450fb71f8f53b3c1ef6327 Gerrit-Change-Number: 14081 Gerrit-PatchSet: 3 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Anonymous Coward (521) Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Fri, 16 Aug 2019 19:37:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8866: Bump versions of dependencies
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14083 ) Change subject: IMPALA-8866: Bump versions of dependencies .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4285/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14083 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia60d1ffec034790205404550c4bac54b247547d7 Gerrit-Change-Number: 14083 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 16 Aug 2019 19:35:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8124: Modify TestWebPage::test catalog to avoid flakiness.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14075 ) Change subject: IMPALA-8124: Modify TestWebPage::test_catalog to avoid flakiness. .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4284/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14075 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I341bf25baf8d9316a21a9eff860de84b33afd12f Gerrit-Change-Number: 14075 Gerrit-PatchSet: 2 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 16 Aug 2019 19:32:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8847: Fix test event processing.py on Hive-3
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14082 ) Change subject: IMPALA-8847: Fix test_event_processing.py on Hive-3 .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4283/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14082 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6507540bfbc0d131a061865ed9ed94792ccfa758 Gerrit-Change-Number: 14082 Gerrit-PatchSet: 1 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 16 Aug 2019 19:29:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7935: Disable /catalog object in local catalog mode.
Anurag Mantripragada has posted comments on this change. (
http://gerrit.cloudera.org:8080/12443 )
Change subject: IMPALA-7935: Disable /catalog_object in local catalog mode.
..
Patch Set 9:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/12443/8/tests/custom_cluster/test_local_catalog.py
File tests/custom_cluster/test_local_catalog.py:
http://gerrit.cloudera.org:8080/#/c/12443/8/tests/custom_cluster/test_local_catalog.py@386
PS8, Line 386: in impalad.service.read_debug_webpage('/catalog_object')
> nit: 4 indent spaces.
Done
http://gerrit.cloudera.org:8080/#/c/12443/8/www/catalog.tmpl
File www/catalog.tmpl:
http://gerrit.cloudera.org:8080/#/c/12443/8/www/catalog.tmpl@42
PS8, Line 42:
[Impala-ASF-CR] IMPALA-7935: Disable /catalog object in local catalog mode.
Anurag Mantripragada has uploaded a new patch set (#9). ( http://gerrit.cloudera.org:8080/12443 ) Change subject: IMPALA-7935: Disable /catalog_object in local catalog mode. .. IMPALA-7935: Disable /catalog_object in local catalog mode. getTCatalogObject() is not supported in local catalog mode since metadata is partially fetched on demand. Removed hyperlinks to the /catalog_object endpoints when local_catalog_mode is enabled. Testing: Added a test to test_local_catalog::TestObservability to verify /catalog_mode endpoint is disabled when in local catalog mode. Change-Id: Ia04797b32964c2edaa2e860dcf510d6f9cccd81c --- M be/src/service/impala-http-handler.cc M tests/custom_cluster/test_local_catalog.py M www/catalog.tmpl 3 files changed, 45 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/12443/9 -- To view, visit http://gerrit.cloudera.org:8080/12443 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia04797b32964c2edaa2e860dcf510d6f9cccd81c Gerrit-Change-Number: 12443 Gerrit-PatchSet: 9 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8793: Implement TRUNCATE for insert-only ACID tables
Csaba Ringhofer has posted comments on this change. (
http://gerrit.cloudera.org:8080/14071 )
Change subject: IMPALA-8793: Implement TRUNCATE for insert-only ACID tables
..
Patch Set 2:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/14071/2/fe/src/main/java/org/apache/impala/catalog/Catalog.java
File fe/src/main/java/org/apache/impala/catalog/Catalog.java:
http://gerrit.cloudera.org:8080/#/c/14071/2/fe/src/main/java/org/apache/impala/catalog/Catalog.java@732
PS2, Line 732: 0L
Can this be called with txnId == -1? I would change this to <=0 or add a
precondition for 0<=txnId
http://gerrit.cloudera.org:8080/#/c/14071/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java:
http://gerrit.cloudera.org:8080/#/c/14071/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1800
PS2, Line 1800: createion
typo: creation
http://gerrit.cloudera.org:8080/#/c/14071/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1829
PS2, Line 1829: if (hmsPart.getParameters() != null) {
:
hmsPart.getParameters().remove(StatsSetupConst.COLUMN_STATS_ACCURATE);
Do we have to alter the partition if COLUMN_STATS_ACCURATE was set? In a table
that is generally written by Impala it will be never set and we could spare the
alter partitions RPC. This optimization was implemented for INSERT in
https://gerrit.cloudera.org/#/c/14037/8/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
http://gerrit.cloudera.org:8080/#/c/14071/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1838
PS2, Line 1838: unsetPartitionsColStats(table.getMetaStoreTable(),
hmsPartitions,
: writeId, txn.getId());
: }
: // Remove COLUMN_STATS_ACCURATE property from the table.
: unsetTableColStats(table.getMetaStoreTable(),
txn.getId()
Note that this will break when https://gerrit.cloudera.org/#/c/14066 is merged.
http://gerrit.cloudera.org:8080/#/c/14071/2/testdata/workloads/functional-query/queries/QueryTest/acid-truncate.test
File testdata/workloads/functional-query/queries/QueryTest/acid-truncate.test:
http://gerrit.cloudera.org:8080/#/c/14071/2/testdata/workloads/functional-query/queries/QueryTest/acid-truncate.test@96
PS2, Line 96: truncate table pt;
Shouldn't this have a RESULTS section?
--
To view, visit http://gerrit.cloudera.org:8080/14071
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic749b7f27da157e1c0ebf9b7e9b6ee09afad122a
Gerrit-Change-Number: 14071
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Dinesh Garg (430)
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong
Gerrit-Reviewer: Vihang Karajgaonkar
Gerrit-Reviewer: Yongzhi Chen
Gerrit-Reviewer: Zoltan Borok-Nagy
Gerrit-Comment-Date: Fri, 16 Aug 2019 19:16:41 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7935: Disable /catalog object in local catalog mode.
Bharath Vissapragada has posted comments on this change. (
http://gerrit.cloudera.org:8080/12443 )
Change subject: IMPALA-7935: Disable /catalog_object in local catalog mode.
..
Patch Set 8: Code-Review+2
(2 comments)
http://gerrit.cloudera.org:8080/#/c/12443/8/tests/custom_cluster/test_local_catalog.py
File tests/custom_cluster/test_local_catalog.py:
http://gerrit.cloudera.org:8080/#/c/12443/8/tests/custom_cluster/test_local_catalog.py@386
PS8, Line 386: in
impalad.service.read_debug_webpage('/catalog_object')
nit: 4 indent spaces.
http://gerrit.cloudera.org:8080/#/c/12443/8/www/catalog.tmpl
File www/catalog.tmpl:
http://gerrit.cloudera.org:8080/#/c/12443/8/www/catalog.tmpl@42
PS8, Line 42: {{?use_local_catalog}} {{name}} {{/use_local_catalog}}
nit: Add a quick comment that /catalog_object is disabled in local catalog mode?
--
To view, visit http://gerrit.cloudera.org:8080/12443
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia04797b32964c2edaa2e860dcf510d6f9cccd81c
Gerrit-Change-Number: 12443
Gerrit-PatchSet: 8
Gerrit-Owner: Anurag Mantripragada
Gerrit-Reviewer: Anurag Mantripragada
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Fri, 16 Aug 2019 18:50:49 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8841: Try to fix Tez related dataload flakiness
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14081 ) Change subject: IMPALA-8841: Try to fix Tez related dataload flakiness .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14081 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id21d57483fe7a4f72f450fb71f8f53b3c1ef6327 Gerrit-Change-Number: 14081 Gerrit-PatchSet: 3 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Anonymous Coward (521) Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Fri, 16 Aug 2019 18:45:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8871: Upgrade Thrift version in fe
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14084 ) Change subject: IMPALA-8871: Upgrade Thrift version in fe .. Patch Set 1: Code-Review+2 Thanks for doing this -- To view, visit http://gerrit.cloudera.org:8080/14084 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If62e6330f0a725ebd4213c200656fc12a0eccf06 Gerrit-Change-Number: 14084 Gerrit-PatchSet: 1 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 16 Aug 2019 18:39:35 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8871: Upgrade Thrift version in fe
Sahil Takiar has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14084 Change subject: IMPALA-8871: Upgrade Thrift version in fe .. IMPALA-8871: Upgrade Thrift version in fe Upgrade fe/ Thrift version to 0.9.3-1. Testing: * Ran exhaustive tests. * Ran perf-AB-test, which showed no significant perf difference. Change-Id: If62e6330f0a725ebd4213c200656fc12a0eccf06 --- M impala-parent/pom.xml 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/14084/1 -- To view, visit http://gerrit.cloudera.org:8080/14084 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: If62e6330f0a725ebd4213c200656fc12a0eccf06 Gerrit-Change-Number: 14084 Gerrit-PatchSet: 1 Gerrit-Owner: Sahil Takiar
[Impala-ASF-CR] IMPALA-8124: Modify TestWebPage::test catalog to avoid flakiness.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14075 ) Change subject: IMPALA-8124: Modify TestWebPage::test_catalog to avoid flakiness. .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4810/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/14075 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I341bf25baf8d9316a21a9eff860de84b33afd12f Gerrit-Change-Number: 14075 Gerrit-PatchSet: 2 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 16 Aug 2019 18:35:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8124: Modify TestWebPage::test catalog to avoid flakiness.
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/14075 ) Change subject: IMPALA-8124: Modify TestWebPage::test_catalog to avoid flakiness. .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14075 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I341bf25baf8d9316a21a9eff860de84b33afd12f Gerrit-Change-Number: 14075 Gerrit-PatchSet: 2 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 16 Aug 2019 18:35:46 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8584: Add cookie support to the HTTP HS2 server
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13672 ) Change subject: IMPALA-8584: Add cookie support to the HTTP HS2 server .. Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/13672/5/be/src/util/openssl-util.cc File be/src/util/openssl-util.cc: http://gerrit.cloudera.org:8080/#/c/13672/5/be/src/util/openssl-util.cc@169 PS5, Line 169: uint8_t* result = HMAC(EVP_sha256(), key_, KEY_LEN, data, len, out, _len); I think we might need to clean for and clean errors after the HMAC call. Hard to know with the insane OpenSSL APIs. -- To view, visit http://gerrit.cloudera.org:8080/13672 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I647c06f94ef91aa3b6413e91576c4ec506ed57f4 Gerrit-Change-Number: 13672 Gerrit-PatchSet: 5 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 16 Aug 2019 18:19:04 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8866: Bump versions of dependencies
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14083 ) Change subject: IMPALA-8866: Bump versions of dependencies .. Patch Set 2: I did a dumb thing and didn't actually bump jackson-databind. Can you look again? -- To view, visit http://gerrit.cloudera.org:8080/14083 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia60d1ffec034790205404550c4bac54b247547d7 Gerrit-Change-Number: 14083 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 16 Aug 2019 18:14:16 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8866: Bump versions of dependencies
Hello Lars Volker, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14083 to look at the new patch set (#2). Change subject: IMPALA-8866: Bump versions of dependencies .. IMPALA-8866: Bump versions of dependencies Bumped derby to next patch version, 10.14.2.0 Bumped jackson-databind to next patch version, 2.9.9.2 Bump jquery to 3.4.1, the most recent release. Testing: Manually inspected debug page, clicked through most of the screens to sanity check them. Ran exhaustive tests with Hive 2 and Hive 3 configs. Change-Id: Ia60d1ffec034790205404550c4bac54b247547d7 --- M .gitattributes M bin/rat_exclude_files.txt M fe/pom.xml M impala-parent/pom.xml M www/common-header.tmpl D www/jquery/jquery-1.12.4.min.js A www/jquery/jquery-3.4.1.min.js 7 files changed, 19 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/14083/2 -- To view, visit http://gerrit.cloudera.org:8080/14083 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia60d1ffec034790205404550c4bac54b247547d7 Gerrit-Change-Number: 14083 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker
[Impala-ASF-CR] IMPALA-7935: Disable /catalog object in local catalog mode.
Anurag Mantripragada has uploaded a new patch set (#8). ( http://gerrit.cloudera.org:8080/12443 ) Change subject: IMPALA-7935: Disable /catalog_object in local catalog mode. .. IMPALA-7935: Disable /catalog_object in local catalog mode. getTCatalogObject() is not supported in local catalog mode since metadata is partially fetched on demand. Removed hyperlinks to the /catalog_object endpoints when local_catalog_mode is enabled. Testing: Added a test to test_local_catalog::TestObservability to verify /catalog_mode endpoint is disabled when in local catalog mode. Change-Id: Ia04797b32964c2edaa2e860dcf510d6f9cccd81c --- M be/src/service/impala-http-handler.cc M tests/custom_cluster/test_local_catalog.py M www/catalog.tmpl 3 files changed, 43 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/12443/8 -- To view, visit http://gerrit.cloudera.org:8080/12443 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia04797b32964c2edaa2e860dcf510d6f9cccd81c Gerrit-Change-Number: 12443 Gerrit-PatchSet: 8 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-7935: Disable /catalog object in local catalog mode.
Anurag Mantripragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/12443 ) Change subject: IMPALA-7935: Disable /catalog_object in local catalog mode. .. Patch Set 8: (2 comments) http://gerrit.cloudera.org:8080/#/c/12443/7/be/src/service/impala-http-handler.cc File be/src/service/impala-http-handler.cc: http://gerrit.cloudera.org:8080/#/c/12443/7/be/src/service/impala-http-handler.cc@109 PS7, Line 109: // The /catalog_object endpoint is disabled if local_catalog_mode is used > nit: add why Done. http://gerrit.cloudera.org:8080/#/c/12443/7/tests/webserver/test_web_pages.py File tests/webserver/test_web_pages.py: http://gerrit.cloudera.org:8080/#/c/12443/7/tests/webserver/test_web_pages.py@610 PS7, Line 610: > Can you also scrape the webpage source and make sure it doesn't have a "cat In fact, this test is redundant. There is already a test for checking /catalog_obj endpoint is working for both loaded and unloaded tables. Removing this. -- To view, visit http://gerrit.cloudera.org:8080/12443 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia04797b32964c2edaa2e860dcf510d6f9cccd81c Gerrit-Change-Number: 12443 Gerrit-PatchSet: 8 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 16 Aug 2019 18:12:11 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8841: Try to fix Tez related dataload flakiness
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/14081 ) Change subject: IMPALA-8841: Try to fix Tez related dataload flakiness .. Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/14081/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/14081/2//COMMIT_MSG@10 PS2, Line 10: initializing Tez resources in parallel > my observation is that the tez resources are allocated per session and sinc My experience is that the .jar in question is not overwritten by new sessions if the file already exists, so I have some hopes about helping with this change. http://gerrit.cloudera.org:8080/#/c/14081/2/testdata/bin/create-load-data.sh File testdata/bin/create-load-data.sh: http://gerrit.cloudera.org:8080/#/c/14081/2/testdata/bin/create-load-data.sh@620 PS2, Line 620: before before > typo Done -- To view, visit http://gerrit.cloudera.org:8080/14081 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id21d57483fe7a4f72f450fb71f8f53b3c1ef6327 Gerrit-Change-Number: 14081 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Anonymous Coward (521) Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Fri, 16 Aug 2019 18:00:45 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8866: Bump versions of dependencies
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/14083 ) Change subject: IMPALA-8866: Bump versions of dependencies .. Patch Set 1: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/14083/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/14083/1//COMMIT_MSG@9 PS1, Line 9: Bumped derby to next patch version. nit: I think it would be great to include the versions in the commit message, too -- To view, visit http://gerrit.cloudera.org:8080/14083 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia60d1ffec034790205404550c4bac54b247547d7 Gerrit-Change-Number: 14083 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Comment-Date: Fri, 16 Aug 2019 17:55:41 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8836: Support COMPUTE STATS on insert only ACID tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14066 ) Change subject: IMPALA-8836: Support COMPUTE STATS on insert only ACID tables .. Patch Set 8: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4282/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14066 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5c06b4678c1ff75c5aa1586a78afea563e64057f Gerrit-Change-Number: 14066 Gerrit-PatchSet: 8 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 16 Aug 2019 17:42:13 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8841: Try to fix Tez related dataload flakiness
Hello Vihang Karajgaonkar, Anonymous Coward (521), Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14081 to look at the new patch set (#3). Change subject: IMPALA-8841: Try to fix Tez related dataload flakiness .. IMPALA-8841: Try to fix Tez related dataload flakiness The flakiness may be related to starting Hive queries in parallel which triggers initializing Tez resources in parallel (only needed at the first statement that uses Tez). Doing a non-parallel statement at first may solve the issue. Also includes a fix for a recent issue in 'build-and-copy-hive-udfs' introduced by the version bump in https://gerrit.cloudera.org/#/c/14043/ Change-Id: Id21d57483fe7a4f72f450fb71f8f53b3c1ef6327 --- M testdata/bin/create-load-data.sh M tests/test-hive-udfs/pom.xml 2 files changed, 19 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/81/14081/3 -- To view, visit http://gerrit.cloudera.org:8080/14081 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id21d57483fe7a4f72f450fb71f8f53b3c1ef6327 Gerrit-Change-Number: 14081 Gerrit-PatchSet: 3 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Anonymous Coward (521) Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8866: Bump versions of dependencies
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14083 ) Change subject: IMPALA-8866: Bump versions of dependencies .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4809/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/14083 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia60d1ffec034790205404550c4bac54b247547d7 Gerrit-Change-Number: 14083 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 16 Aug 2019 17:50:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8866: Bump versions of dependencies
Tim Armstrong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14083 Change subject: IMPALA-8866: Bump versions of dependencies .. IMPALA-8866: Bump versions of dependencies Bumped derby to next patch version. Bumped jackson-databind to next patch version Bump jquery to 3.4.1, the most recent release. Testing: Manually inspected debug page, clicked through most of the screens to sanity check them. Ran exhaustive tests with Hive 2 and Hive 3 configs. Change-Id: Ia60d1ffec034790205404550c4bac54b247547d7 --- M .gitattributes M bin/rat_exclude_files.txt M fe/pom.xml M www/common-header.tmpl D www/jquery/jquery-1.12.4.min.js A www/jquery/jquery-3.4.1.min.js 6 files changed, 11 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/14083/1 -- To view, visit http://gerrit.cloudera.org:8080/14083 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ia60d1ffec034790205404550c4bac54b247547d7 Gerrit-Change-Number: 14083 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong
[Impala-ASF-CR] IMPALA-8124: Modify TestWebPage::test catalog to avoid flakiness.
Anurag Mantripragada has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/14075 ) Change subject: IMPALA-8124: Modify TestWebPage::test_catalog to avoid flakiness. .. IMPALA-8124: Modify TestWebPage::test_catalog to avoid flakiness. This test scrapes the /catalog webpage for metrics. This can occasionally run into a race condition when a table lock is being held for functional*.* tables and the test tries to get the metrics. This result in a failed assert for the metrics. This change rewrites the test to create new tables with unique_database fixture to avoid flakiness. Also, seperated out an assert with AND into two different asserts so it's easy to track which one failed. Testing: Ran multiple runs of this test locally while refreshing the table. Change-Id: I341bf25baf8d9316a21a9eff860de84b33afd12f --- M tests/webserver/test_web_pages.py 1 file changed, 41 insertions(+), 14 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/75/14075/2 -- To view, visit http://gerrit.cloudera.org:8080/14075 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I341bf25baf8d9316a21a9eff860de84b33afd12f Gerrit-Change-Number: 14075 Gerrit-PatchSet: 2 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8847: Fix test event processing.py on Hive-3
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14082 ) Change subject: IMPALA-8847: Fix test_event_processing.py on Hive-3 .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4808/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/14082 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6507540bfbc0d131a061865ed9ed94792ccfa758 Gerrit-Change-Number: 14082 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 16 Aug 2019 17:37:01 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8847: Fix test event processing.py on Hive-3
Hello Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14082 to look at the new patch set (#2). Change subject: IMPALA-8847: Fix test_event_processing.py on Hive-3 .. IMPALA-8847: Fix test_event_processing.py on Hive-3 The test_event_processing fails on Hive-3 due to a error in the syntax of the hive query for transactional tables. It was missed unfortunately when IMPALA-8847 was merged. Testing done: Ran the test on both Hive-2 and Hive-3 and it passes now Change-Id: I6507540bfbc0d131a061865ed9ed94792ccfa758 --- M tests/custom_cluster/test_event_processing.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/82/14082/2 -- To view, visit http://gerrit.cloudera.org:8080/14082 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6507540bfbc0d131a061865ed9ed94792ccfa758 Gerrit-Change-Number: 14082 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] Fix test event processing.py on Hive-3
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/14082 ) Change subject: Fix test_event_processing.py on Hive-3 .. Patch Set 1: Code-Review+2 (2 comments) http://gerrit.cloudera.org:8080/#/c/14082/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/14082/1//COMMIT_MSG@7 PS1, Line 7: Fix test_event_processing.py on Hive-3 I would add the Jira of the original commit. http://gerrit.cloudera.org:8080/#/c/14082/1//COMMIT_MSG@11 PS1, Line 11: IMPALA-8847 was merged nit: missing . -- To view, visit http://gerrit.cloudera.org:8080/14082 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6507540bfbc0d131a061865ed9ed94792ccfa758 Gerrit-Change-Number: 14082 Gerrit-PatchSet: 1 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 16 Aug 2019 17:32:30 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Fix test event processing.py on Hive-3
Vihang Karajgaonkar has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14082 Change subject: Fix test_event_processing.py on Hive-3 .. Fix test_event_processing.py on Hive-3 The test_event_processing fails on Hive-3 due to a error in the syntax of the hive query for transactional tables. It was missed unfortunately when IMPALA-8847 was merged Testing done: Ran the test on both Hive-2 and Hive-3 and it passes now Change-Id: I6507540bfbc0d131a061865ed9ed94792ccfa758 --- M tests/custom_cluster/test_event_processing.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/82/14082/1 -- To view, visit http://gerrit.cloudera.org:8080/14082 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I6507540bfbc0d131a061865ed9ed94792ccfa758 Gerrit-Change-Number: 14082 Gerrit-PatchSet: 1 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Csaba Ringhofer
[Impala-ASF-CR] IMPALA-8793: Implement TRUNCATE for insert-only ACID tables
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14071 ) Change subject: IMPALA-8793: Implement TRUNCATE for insert-only ACID tables .. Patch Set 2: (5 comments) http://gerrit.cloudera.org:8080/#/c/14071/2/fe/src/main/java/org/apache/impala/catalog/Transaction.java File fe/src/main/java/org/apache/impala/catalog/Transaction.java: http://gerrit.cloudera.org:8080/#/c/14071/2/fe/src/main/java/org/apache/impala/catalog/Transaction.java@17 PS2, Line 17: Transaction Does this class need to be thread-safe? I can see there are possible races with respect to the transactionId_ being used as the flag to track if this transaction is open or close. http://gerrit.cloudera.org:8080/#/c/14071/2/fe/src/main/java/org/apache/impala/catalog/Transaction.java@20 PS2, Line 20: transactionId_ can we make this final? I see that you are using this to keep track of whether the transaction is open or close. May be you can use a AtomicBoolean to keep track of that separately so that commit() can be made thread-safe. http://gerrit.cloudera.org:8080/#/c/14071/2/fe/src/main/java/org/apache/impala/catalog/Transaction.java@38 PS2, Line 38: checkState(transactionId_ > 0) Does this need to throw an exception? What is the behavior from HMS if a client calls commit on a already committed transaction? http://gerrit.cloudera.org:8080/#/c/14071/2/fe/src/main/java/org/apache/impala/catalog/Transaction.java@46 PS2, Line 46: transactionId_ nit, can we change this to if (transactionId_ < 0) return; makes it easier to read in my opinion. http://gerrit.cloudera.org:8080/#/c/14071/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/14071/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1821 PS2, Line 1821: COLUMN_STATS_ACCURATE Its unclear why we are clearing this property. Can you add a comment explaining the same? -- To view, visit http://gerrit.cloudera.org:8080/14071 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic749b7f27da157e1c0ebf9b7e9b6ee09afad122a Gerrit-Change-Number: 14071 Gerrit-PatchSet: 2 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Dinesh Garg (430) Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 16 Aug 2019 17:22:42 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8836: Support COMPUTE STATS on insert only ACID tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14066 ) Change subject: IMPALA-8836: Support COMPUTE STATS on insert only ACID tables .. Patch Set 9: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4807/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/14066 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5c06b4678c1ff75c5aa1586a78afea563e64057f Gerrit-Change-Number: 14066 Gerrit-PatchSet: 9 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 16 Aug 2019 17:07:01 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8836: Support COMPUTE STATS on insert only ACID tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14066 ) Change subject: IMPALA-8836: Support COMPUTE STATS on insert only ACID tables .. Patch Set 9: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14066 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5c06b4678c1ff75c5aa1586a78afea563e64057f Gerrit-Change-Number: 14066 Gerrit-PatchSet: 9 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 16 Aug 2019 17:07:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8836: Support COMPUTE STATS on insert only ACID tables
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/14066 ) Change subject: IMPALA-8836: Support COMPUTE STATS on insert only ACID tables .. Patch Set 8: Code-Review+2 Carry +2 -- To view, visit http://gerrit.cloudera.org:8080/14066 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5c06b4678c1ff75c5aa1586a78afea563e64057f Gerrit-Change-Number: 14066 Gerrit-PatchSet: 8 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 16 Aug 2019 17:06:38 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8836: Support COMPUTE STATS on insert only ACID tables
Csaba Ringhofer has posted comments on this change. (
http://gerrit.cloudera.org:8080/14066 )
Change subject: IMPALA-8836: Support COMPUTE STATS on insert only ACID tables
..
Patch Set 7:
(9 comments)
http://gerrit.cloudera.org:8080/#/c/14066/7//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/14066/7//COMMIT_MSG@9
PS7, Line 9: For ACID tables COMPUTE STATS needs to use a new HMS API, as the
> You could also mention the inter-operability aspects with Hive, e.g. will H
I added a paragraph to the additional change related to COLUMN_STATS_ACCURATE.
I don't think that there are any other changes.
http://gerrit.cloudera.org:8080/#/c/14066/7//COMMIT_MSG@32
PS7, Line 32: l
> nit: typo
Done
http://gerrit.cloudera.org:8080/#/c/14066/7/fe/src/compat-hive-2/java/org/apache/impala/compat/MetastoreShim.java
File fe/src/compat-hive-2/java/org/apache/impala/compat/MetastoreShim.java:
http://gerrit.cloudera.org:8080/#/c/14066/7/fe/src/compat-hive-2/java/org/apache/impala/compat/MetastoreShim.java@79
PS7, Line 79: TblTransaction
> nit: it's a bit weird that Tbl is abbreviated while Transaction isn't
My reasoning was that Tbl is self-evident in Impala context, while Txn is only
self-evident for ACID people. I am not happy about the name, but I plan to
redesign the class anyway.
http://gerrit.cloudera.org:8080/#/c/14066/7/fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java
File fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java:
http://gerrit.cloudera.org:8080/#/c/14066/7/fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java@238
PS7, Line 238: for (Partition part: partitions) {
> nit: missing space
Done
http://gerrit.cloudera.org:8080/#/c/14066/7/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java:
http://gerrit.cloudera.org:8080/#/c/14066/7/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@948
PS7, Line 948: // TODO: Transaction committing / aborting seems weird for
stat update, but I don't
: // see other ways to get a new write id (which is
needed to update
: // transactional tables). Hive seems to use internal
API for this.
> nit: so what should we do here? Is there a Jira here to track?
Done
http://gerrit.cloudera.org:8080/#/c/14066/7/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@956
PS7, Line 956: -1 /* opens new transaction */
> nit: The interface is confusing. Later we should cut createTblTransaction()
I agree, but I would do this later when the whole TblTransaction thing is
redesigned.
http://gerrit.cloudera.org:8080/#/c/14066/7/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@3391
PS7, Line 3391: //
> Should we remove this line and the comment above?
Thanks for catching this, this was commented out as an experiment and I forgot
to undo it.
http://gerrit.cloudera.org:8080/#/c/14066/7/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@3575
PS7, Line 3575:
> nit too much spaces
Done
http://gerrit.cloudera.org:8080/#/c/14066/7/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@3579
PS7, Line 3579:
> nit: too much spaces
Done
--
To view, visit http://gerrit.cloudera.org:8080/14066
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5c06b4678c1ff75c5aa1586a78afea563e64057f
Gerrit-Change-Number: 14066
Gerrit-PatchSet: 7
Gerrit-Owner: Csaba Ringhofer
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Yongzhi Chen
Gerrit-Reviewer: Zoltan Borok-Nagy
Gerrit-Comment-Date: Fri, 16 Aug 2019 17:06:24 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8836: Support COMPUTE STATS on insert only ACID tables
Hello Yongzhi Chen, Zoltan Borok-Nagy, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14066 to look at the new patch set (#8). Change subject: IMPALA-8836: Support COMPUTE STATS on insert only ACID tables .. IMPALA-8836: Support COMPUTE STATS on insert only ACID tables For ACID tables COMPUTE STATS needs to use a new HMS API, as the old one is rejected by metastore. This API currently has some counter intuitive parts: - setPartitionColumnStatistics is used to set table stats, as there is no similar function exposed by HMS client for tables at the moment. - A new writeId is allocated for the stat change, and this needs a transaction, so a transaction is opened/committed/aborted even though this doesn't seem necessary. The Hive code seems to use internal API for this. - Even though the HMS thrift Table object has a colStats field, it is only applied during alter_table if there are other changes like new columns in the tables, so alter_table couldn't be used to change column stats. Additional changes: - DROP STATS is no longer allowed for transactional tables, as it turned out that there is no transactional version of the old API. - Remove COLUMN_STATS_ACCURATE table property during COMPUTE STATS to ensure that Hive does use stats computed by Impala to return answer queries like SELECT count(*) - Changed CatalogOpExecutor.updateCatalog() to get the writeIds earlier. This can mean unnecassary HMS RPC calls if no property change is needed in the end, but I felt it hard to reason about what happens if these RPC calls fail at their original location. TODOs (My plan is to do these in IMPALA-8865): - Tried to make the MetastoreShim API easier to use by adding a class to encapsulate thing like txnId and writeId, but it feels rather half baked and under documented. A similar class is added in https://gerrit.cloudera.org/#/c/14071/, it would be good to merge them. - The validWriteIdList of the original SELECT(s) behind COMPUTE STATS could be used in the HMS API calls, but this would need more plumbing. Change-Id: I5c06b4678c1ff75c5aa1586a78afea563e64057f --- M fe/src/compat-hive-2/java/org/apache/impala/compat/MetastoreShim.java M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java M fe/src/main/java/org/apache/impala/analysis/DropStatsStmt.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/test/java/org/apache/impala/analysis/AnalyzerTest.java A testdata/workloads/functional-query/queries/QueryTest/acid-compute-stats.test M testdata/workloads/functional-query/queries/QueryTest/acid-negative.test M tests/query_test/test_acid.py 10 files changed, 405 insertions(+), 119 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/66/14066/8 -- To view, visit http://gerrit.cloudera.org:8080/14066 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5c06b4678c1ff75c5aa1586a78afea563e64057f Gerrit-Change-Number: 14066 Gerrit-PatchSet: 8 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-8841: Try to fix Tez related dataload flakiness
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14081 ) Change subject: IMPALA-8841: Try to fix Tez related dataload flakiness .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4281/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14081 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id21d57483fe7a4f72f450fb71f8f53b3c1ef6327 Gerrit-Change-Number: 14081 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Anonymous Coward (521) Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Fri, 16 Aug 2019 16:54:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8790: fix referencing wrong grouping exprs of MultiAggregateInfo
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/14063 ) Change subject: IMPALA-8790: fix referencing wrong grouping exprs of MultiAggregateInfo .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14063 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I11a80bd6d73ea00ad8c644469558a1885706f596 Gerrit-Change-Number: 14063 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Fri, 16 Aug 2019 16:41:01 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8841: Try to fix Tez related dataload flakiness
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14081 ) Change subject: IMPALA-8841: Try to fix Tez related dataload flakiness .. Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/14081/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/14081/2//COMMIT_MSG@10 PS2, Line 10: initializing Tez resources in parallel my observation is that the tez resources are allocated per session and since every hive query which is run during dataload happens in a separate beeline session, it may still need to initialize the resources. Right? http://gerrit.cloudera.org:8080/#/c/14081/2/testdata/bin/create-load-data.sh File testdata/bin/create-load-data.sh: http://gerrit.cloudera.org:8080/#/c/14081/2/testdata/bin/create-load-data.sh@620 PS2, Line 620: before before typo -- To view, visit http://gerrit.cloudera.org:8080/14081 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id21d57483fe7a4f72f450fb71f8f53b3c1ef6327 Gerrit-Change-Number: 14081 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Anonymous Coward (521) Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Fri, 16 Aug 2019 16:35:19 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8661 : Add randomized tests to stress MetastoreEventsProcessor
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13932 ) Change subject: IMPALA-8661 : Add randomized tests to stress MetastoreEventsProcessor .. Patch Set 9: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4806/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/13932 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8c85b83efd4f56b5ae0e8d1dc6a2ee2feb6721ce Gerrit-Change-Number: 13932 Gerrit-PatchSet: 9 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Fri, 16 Aug 2019 16:27:16 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8841: Try to fix Tez related dataload flakiness
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14081 ) Change subject: IMPALA-8841: Try to fix Tez related dataload flakiness .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4805/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/14081 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id21d57483fe7a4f72f450fb71f8f53b3c1ef6327 Gerrit-Change-Number: 14081 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Anonymous Coward (521) Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 16 Aug 2019 16:24:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8836: Support COMPUTE STATS on insert only ACID tables
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/14066 ) Change subject: IMPALA-8836: Support COMPUTE STATS on insert only ACID tables .. Patch Set 7: Code-Review+2 +2ed it so you can carry it if you fix those nits -- To view, visit http://gerrit.cloudera.org:8080/14066 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5c06b4678c1ff75c5aa1586a78afea563e64057f Gerrit-Change-Number: 14066 Gerrit-PatchSet: 7 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 16 Aug 2019 16:21:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8793: Implement TRUNCATE for insert-only ACID tables
Zoltan Borok-Nagy has removed a vote on this change. Change subject: IMPALA-8793: Implement TRUNCATE for insert-only ACID tables .. Removed Code-Review+2 by Zoltan Borok-Nagy -- To view, visit http://gerrit.cloudera.org:8080/14071 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: Ic749b7f27da157e1c0ebf9b7e9b6ee09afad122a Gerrit-Change-Number: 14071 Gerrit-PatchSet: 2 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Dinesh Garg (430) Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-8841: Try to fix Tez related dataload flakiness
Hello Anonymous Coward (521), Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14081 to look at the new patch set (#2). Change subject: IMPALA-8841: Try to fix Tez related dataload flakiness .. IMPALA-8841: Try to fix Tez related dataload flakiness The flakiness may be related to starting Hive queries in parallel which triggers initializing Tez resources in parallel (only needed at the first statement that uses Tez). Doing a non-parallel statement at first may solve the issue. Also includes a fix for a recent issue in 'build-and-copy-hive-udfs' introduced by the version bump in https://gerrit.cloudera.org/#/c/14043/ Change-Id: Id21d57483fe7a4f72f450fb71f8f53b3c1ef6327 --- M testdata/bin/create-load-data.sh M tests/test-hive-udfs/pom.xml 2 files changed, 20 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/81/14081/2 -- To view, visit http://gerrit.cloudera.org:8080/14081 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id21d57483fe7a4f72f450fb71f8f53b3c1ef6327 Gerrit-Change-Number: 14081 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Anonymous Coward (521) Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8793: Implement TRUNCATE for insert-only ACID tables
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/14071 ) Change subject: IMPALA-8793: Implement TRUNCATE for insert-only ACID tables .. Patch Set 2: Code-Review+2 +2ed it so you can carry it if you fix those nits -- To view, visit http://gerrit.cloudera.org:8080/14071 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic749b7f27da157e1c0ebf9b7e9b6ee09afad122a Gerrit-Change-Number: 14071 Gerrit-PatchSet: 2 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Dinesh Garg (430) Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 16 Aug 2019 16:03:08 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8862: Don't ship jetty and ant
Tim Armstrong has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14068 ) Change subject: IMPALA-8862: Don't ship jetty and ant .. IMPALA-8862: Don't ship jetty and ant We shouldn't need the jetty server or the ant build tool at runtime (except possibly jetty utils used by hadoop-yarn-common). Add exclusions and also add these to maven enforcer checks to prevent them creeping back in. Also remove the commons-codec dependency since it isn't used in Impala and we don't need to pin the version to an old one (1.11 is pulled in via hadoop-hdfs). Testing: Built against both CDH and CDP (i.e. Hive 2 and 3). Change-Id: I42c50da957e3704c919b391ecd03c26ed851bdaf Reviewed-on: http://gerrit.cloudera.org:8080/14068 Tested-by: Impala Public Jenkins Reviewed-by: Tim Armstrong --- M fe/pom.xml 1 file changed, 85 insertions(+), 6 deletions(-) Approvals: Impala Public Jenkins: Verified Tim Armstrong: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/14068 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I42c50da957e3704c919b391ecd03c26ed851bdaf Gerrit-Change-Number: 14068 Gerrit-PatchSet: 7 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8862: Don't ship jetty and ant
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14068 ) Change subject: IMPALA-8862: Don't ship jetty and ant .. Patch Set 6: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14068 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I42c50da957e3704c919b391ecd03c26ed851bdaf Gerrit-Change-Number: 14068 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Fri, 16 Aug 2019 16:02:59 + Gerrit-HasComments: No
