[Impala-ASF-CR] IMPALA-8586: Support download URLs for CDP
Joe McDonnell has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13432 ) Change subject: IMPALA-8586: Support download URLs for CDP .. IMPALA-8586: Support download URLs for CDP bin/bootstrap_toolchain.py has accumulated complexity over time. CDH, CDP, and the native toolchain all use different download machinery and naming. One feature that is needed on the CDP side is the ability to specify the download URL in an IMPALA_*_URL environment variable. This adds that support and refactors CDH and native toolchain downloads to use the new system. This is essentially a rewrite of bin/bootstrap_toolchain.py. Currently, there are multiple phases of downloads, each with their own download functions and peculiarities to account for package names and destinations for downloads. This changes the logic so that a package will generate a DownloadUnpackTarball that is completely resolved. It contains everything about what to download and where to put it as well as a needs_download() function and a download() function. Once there is a list of DownloadUnpackTarball objects, they can all be downloaded and unpacked in a single phase. This implements different types of packages as subclasses of DownloadUnpackTarball. Since most subclasses want to be able to construct URLs and archive names using templates, the TemplatedDownloadUnpackTarball takes the same arguments as DownloadUnpackTarball along with a map of template substitutions, which are applied to all string arguments. Kudu requires special handling and gets its own set of subclasses to handle various subtleties like toolchain vs CDH Kudu, the Kudu stub, and making sure that the "kudu" package and the "kudu-java" package don't confuse each other. As part of this change, USE_CDP_HIVE=true now uses the CDP version of HBase rather than always using the CDH version. Change-Id: I67824fd82b820e68e9f5c87939ec94ca6abadb8c Reviewed-on: http://gerrit.cloudera.org:8080/13432 Reviewed-by: Tim Armstrong Tested-by: Impala Public Jenkins --- M bin/bootstrap_toolchain.py M bin/impala-config.sh M impala-parent/pom.xml 3 files changed, 522 insertions(+), 366 deletions(-) Approvals: Tim Armstrong: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/13432 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I67824fd82b820e68e9f5c87939ec94ca6abadb8c Gerrit-Change-Number: 13432 Gerrit-PatchSet: 5 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-8935: Add links to other daemons from webui
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14215 ) Change subject: IMPALA-8935: Add links to other daemons from webui .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/14215 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4f92ed47ea3d96dbb8837f23467906510aec2e78 Gerrit-Change-Number: 14215 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 13 Sep 2019 01:38:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8935: Add links to other daemons from webui
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14215 ) Change subject: IMPALA-8935: Add links to other daemons from webui .. IMPALA-8935: Add links to other daemons from webui This patch adds links on the /backends page of the webui to the webuis for the other backends. Testing: - Updated test_backends_page to check for the webui links. Change-Id: I4f92ed47ea3d96dbb8837f23467906510aec2e78 Reviewed-on: http://gerrit.cloudera.org:8080/14215 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/service/impala-http-handler.cc M tests/webserver/test_web_pages.py M www/backends.tmpl 3 files changed, 11 insertions(+), 0 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/14215 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I4f92ed47ea3d96dbb8837f23467906510aec2e78 Gerrit-Change-Number: 14215 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8586: Support download URLs for CDP
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13432 ) Change subject: IMPALA-8586: Support download URLs for CDP .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13432 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I67824fd82b820e68e9f5c87939ec94ca6abadb8c Gerrit-Change-Number: 13432 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 13 Sep 2019 01:33:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8932: addendum - protocol var not defined
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14225 ) Change subject: IMPALA-8932: addendum - protocol var not defined .. Patch Set 2: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4944/ -- To view, visit http://gerrit.cloudera.org:8080/14225 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I75c41a02bc7f1314e48bb5a39b945119264ce478 Gerrit-Change-Number: 14225 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 12 Sep 2019 23:39:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8932: addendum - protocol var not defined
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14225 ) Change subject: IMPALA-8932: addendum - protocol var not defined .. Patch Set 1: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4943/ -- To view, visit http://gerrit.cloudera.org:8080/14225 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I75c41a02bc7f1314e48bb5a39b945119264ce478 Gerrit-Change-Number: 14225 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 12 Sep 2019 22:34:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7984: Port runtime filter from Thrift RPC to KRPC
Fang-Yu Rao has posted comments on this change. (
http://gerrit.cloudera.org:8080/13882 )
Change subject: IMPALA-7984: Port runtime filter from Thrift RPC to KRPC
..
Patch Set 19:
(1 comment)
There is an additional note added to the comments on Line 1117. Thanks!
http://gerrit.cloudera.org:8080/#/c/13882/18/be/src/runtime/coordinator.cc
File be/src/runtime/coordinator.cc:
http://gerrit.cloudera.org:8080/#/c/13882/18/be/src/runtime/coordinator.cc@1115
PS18, Line 1115: if (!status.ok()) {
: LOG(ERROR) << "Cannot get inbound sidecar: " <<
status.message().ToString();
: Disable(coord->filter_mem_tracker_);
> Thanks very much for the detailed explanation!
I think the observation above actually indicates that even though the Bloom
filter fed to RuntimeFilterBank::PublishGlobalFilter() corresponds to an always
false filter, our current implementation will still call BloomFilter::Init() to
allocate the memory space, which seems like a redundant and wasteful operation
because the allocated Bloom filter is basically a bit array of all 0's. Maybe
we could also simplify the current logic in a follow-up JIRA.
--
To view, visit http://gerrit.cloudera.org:8080/13882
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6b394796d250286510e157ae326882bfc01d387a
Gerrit-Change-Number: 13882
Gerrit-PatchSet: 19
Gerrit-Owner: Fang-Yu Rao
Gerrit-Reviewer: Fang-Yu Rao
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Ho
Gerrit-Reviewer: Thomas Tauber-Marshall
Gerrit-Comment-Date: Thu, 12 Sep 2019 22:33:52 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8935: Add links to other daemons from webui
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14215 ) Change subject: IMPALA-8935: Add links to other daemons from webui .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14215 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4f92ed47ea3d96dbb8837f23467906510aec2e78 Gerrit-Change-Number: 14215 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 12 Sep 2019 21:28:46 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8935: Add links to other daemons from webui
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14215 ) Change subject: IMPALA-8935: Add links to other daemons from webui .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4946/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/14215 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4f92ed47ea3d96dbb8837f23467906510aec2e78 Gerrit-Change-Number: 14215 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 12 Sep 2019 21:28:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8935: Add links to other daemons from webui
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14215 ) Change subject: IMPALA-8935: Add links to other daemons from webui .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14215 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4f92ed47ea3d96dbb8837f23467906510aec2e78 Gerrit-Change-Number: 14215 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 12 Sep 2019 21:26:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8586: Support download URLs for CDP
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13432 ) Change subject: IMPALA-8586: Support download URLs for CDP .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4945/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/13432 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I67824fd82b820e68e9f5c87939ec94ca6abadb8c Gerrit-Change-Number: 13432 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 12 Sep 2019 21:22:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8586: Support download URLs for CDP
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13432 ) Change subject: IMPALA-8586: Support download URLs for CDP .. Patch Set 4: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/13432/3/bin/bootstrap_toolchain.py File bin/bootstrap_toolchain.py: http://gerrit.cloudera.org:8080/#/c/13432/3/bin/bootstrap_toolchain.py@129 PS3, Line 129: > Added a brief description. If it would be useful to have more method commen They're about as close to self explanatory as things can be. I think its ok either way. -- To view, visit http://gerrit.cloudera.org:8080/13432 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I67824fd82b820e68e9f5c87939ec94ca6abadb8c Gerrit-Change-Number: 13432 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 12 Sep 2019 21:20:11 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8830: Fix executor group assignment of coordinator only queries
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14183 ) Change subject: IMPALA-8830: Fix executor group assignment of coordinator only queries .. Patch Set 2: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/4555/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/14183 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8fe098032744aa20bbbe4faddfc67e7a46ce03d5 Gerrit-Change-Number: 14183 Gerrit-PatchSet: 2 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Comment-Date: Thu, 12 Sep 2019 20:51:37 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8830: Fix executor group assignment of coordinator only queries
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/14183 ) Change subject: IMPALA-8830: Fix executor group assignment of coordinator only queries .. Patch Set 2: (1 comment) As discussed offline, added a pseudo empty exec groups that we use only for scheduling coord only queries. I skipped making changes to the AssignmentCtx since it was turning out to be a bit unwieldy. We can take it up as clean up in the future. http://gerrit.cloudera.org:8080/#/c/14183/2/be/src/scheduling/admission-controller.cc File be/src/scheduling/admission-controller.cc: http://gerrit.cloudera.org:8080/#/c/14183/2/be/src/scheduling/admission-controller.cc@a690 PS2, Line 690: : : : : : : : : : : : : : : : : : : : : : like we discussed offline, got rid of these checks that dont make sense for groups. -- To view, visit http://gerrit.cloudera.org:8080/14183 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8fe098032744aa20bbbe4faddfc67e7a46ce03d5 Gerrit-Change-Number: 14183 Gerrit-PatchSet: 2 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Comment-Date: Thu, 12 Sep 2019 20:16:31 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8830: Fix executor group assignment of coordinator only queries
Hello Lars Volker, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14183 to look at the new patch set (#2). Change subject: IMPALA-8830: Fix executor group assignment of coordinator only queries .. IMPALA-8830: Fix executor group assignment of coordinator only queries With this fix, coordinator only queries are submitted to a pseudo executor group named "coordinator-only-group" which is empty. This allows running coordinator only queries regardless of the presence of any healthy executor groups. Testing: Added a custom cluster test and modified tests that relied on coordinator only queries to be queued in absence of executor groups. Change-Id: I8fe098032744aa20bbbe4faddfc67e7a46ce03d5 --- M be/src/scheduling/admission-controller.cc M be/src/scheduling/admission-controller.h M be/src/scheduling/cluster-membership-mgr.cc M be/src/scheduling/cluster-membership-mgr.h M be/src/scheduling/executor-group.cc M be/src/scheduling/scheduler.cc M be/src/scheduling/scheduler.h M tests/custom_cluster/test_executor_groups.py 8 files changed, 95 insertions(+), 76 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/14183/2 -- To view, visit http://gerrit.cloudera.org:8080/14183 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8fe098032744aa20bbbe4faddfc67e7a46ce03d5 Gerrit-Change-Number: 14183 Gerrit-PatchSet: 2 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker
[Impala-ASF-CR] IMPALA-8830: Fix executor group assignment of coordinator only queries
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14183 ) Change subject: IMPALA-8830: Fix executor group assignment of coordinator only queries .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/14183/2/be/src/scheduling/cluster-membership-mgr.h File be/src/scheduling/cluster-membership-mgr.h: http://gerrit.cloudera.org:8080/#/c/14183/2/be/src/scheduling/cluster-membership-mgr.h@177 PS2, Line 177: /// Returns a pointer to the static empty group reserved for scheduling coord only queries. line too long (93 > 90) -- To view, visit http://gerrit.cloudera.org:8080/14183 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8fe098032744aa20bbbe4faddfc67e7a46ce03d5 Gerrit-Change-Number: 14183 Gerrit-PatchSet: 2 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Comment-Date: Thu, 12 Sep 2019 20:12:49 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8932: addendum - protocol var not defined
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14225 ) Change subject: IMPALA-8932: addendum - protocol var not defined .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14225 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I75c41a02bc7f1314e48bb5a39b945119264ce478 Gerrit-Change-Number: 14225 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 12 Sep 2019 19:25:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8932: addendum - protocol var not defined
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14225 ) Change subject: IMPALA-8932: addendum - protocol var not defined .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4944/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/14225 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I75c41a02bc7f1314e48bb5a39b945119264ce478 Gerrit-Change-Number: 14225 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 12 Sep 2019 19:25:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8932: addendum - protocol var not defined
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14225 ) Change subject: IMPALA-8932: addendum - protocol var not defined .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4554/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14225 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I75c41a02bc7f1314e48bb5a39b945119264ce478 Gerrit-Change-Number: 14225 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 12 Sep 2019 19:04:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8548: [DOCS] Impala supports ordinal substitutions in SELECT
Alex Rodoni has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14209 ) Change subject: IMPALA-8548: [DOCS] Impala supports ordinal substitutions in SELECT .. IMPALA-8548: [DOCS] Impala supports ordinal substitutions in SELECT Change-Id: Ia48a4a6711ffcf326f3f51200040420101451967 Reviewed-on: http://gerrit.cloudera.org:8080/14209 Tested-by: Impala Public Jenkins Reviewed-by: Zoltan Borok-Nagy --- M docs/shared/impala_common.xml M docs/topics/impala_select.xml 2 files changed, 11 insertions(+), 21 deletions(-) Approvals: Impala Public Jenkins: Verified Zoltan Borok-Nagy: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/14209 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ia48a4a6711ffcf326f3f51200040420101451967 Gerrit-Change-Number: 14209 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-8548: [DOCS] Impala supports ordinal substitutions in SELECT
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/14209 ) Change subject: IMPALA-8548: [DOCS] Impala supports ordinal substitutions in SELECT .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14209 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia48a4a6711ffcf326f3f51200040420101451967 Gerrit-Change-Number: 14209 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 12 Sep 2019 18:57:46 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8932: addendum - protocol var not defined
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/14225 ) Change subject: IMPALA-8932: addendum - protocol var not defined .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14225 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I75c41a02bc7f1314e48bb5a39b945119264ce478 Gerrit-Change-Number: 14225 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Comment-Date: Thu, 12 Sep 2019 18:24:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8932: addendum - protocol var not defined
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14225 ) Change subject: IMPALA-8932: addendum - protocol var not defined .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4943/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/14225 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I75c41a02bc7f1314e48bb5a39b945119264ce478 Gerrit-Change-Number: 14225 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 12 Sep 2019 18:24:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8932: addendum - protocol var not defined
Tim Armstrong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14225 Change subject: IMPALA-8932: addendum - protocol var not defined .. IMPALA-8932: addendum - protocol var not defined Change-Id: I75c41a02bc7f1314e48bb5a39b945119264ce478 --- M shell/impala_shell.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/25/14225/1 -- To view, visit http://gerrit.cloudera.org:8080/14225 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I75c41a02bc7f1314e48bb5a39b945119264ce478 Gerrit-Change-Number: 14225 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong
[Impala-ASF-CR] IMPALA-8755: Backend support for Z-ordering
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14080 ) Change subject: IMPALA-8755: Backend support for Z-ordering .. Patch Set 7: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/4553/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/14080 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0200748ce3e65ebc5d3530f794c0f80aa335a2ab Gerrit-Change-Number: 14080 Gerrit-PatchSet: 7 Gerrit-Owner: Norbert Luksa Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 12 Sep 2019 14:25:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8755: Backend support for Z-ordering
Impala Public Jenkins has posted comments on this change. (
http://gerrit.cloudera.org:8080/14080 )
Change subject: IMPALA-8755: Backend support for Z-ordering
..
Patch Set 7:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/14080/7/be/src/util/tuple-row-compare-test.cc
File be/src/util/tuple-row-compare-test.cc:
http://gerrit.cloudera.org:8080/#/c/14080/7/be/src/util/tuple-row-compare-test.cc@94
PS7, Line 94: uint8_t* tuple_row_mem =
expr_perm_pool_.Allocate(sizeof(char*) + sizeof(int32_t*) * 2);
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/14080/7/be/src/util/tuple-row-compare.cc
File be/src/util/tuple-row-compare.cc:
http://gerrit.cloudera.org:8080/#/c/14080/7/be/src/util/tuple-row-compare.cc@209
PS7, Line 209: Status TupleRowLexicalComparator::CodegenCompare(LlvmCodeGen*
codegen, llvm::Function** fn) {
line too long (93 > 90)
http://gerrit.cloudera.org:8080/#/c/14080/7/be/src/util/tuple-row-compare.cc@323
PS7, Line 323: constexpr uint128_t mask128 = ((uint128_t) -1) / 2 + 1;
//0x8000;
line too long (95 > 90)
http://gerrit.cloudera.org:8080/#/c/14080/7/be/src/util/tuple-row-compare.cc@395
PS7, Line 395: const uint128_t nanoseconds =
static_cast(ts->time().total_nanoseconds());
line too long (91 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/14080
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0200748ce3e65ebc5d3530f794c0f80aa335a2ab
Gerrit-Change-Number: 14080
Gerrit-PatchSet: 7
Gerrit-Owner: Norbert Luksa
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Norbert Luksa
Gerrit-Reviewer: Zoltan Borok-Nagy
Gerrit-Comment-Date: Thu, 12 Sep 2019 13:46:02 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8755: Backend support for Z-ordering
Norbert Luksa has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14080 Change subject: IMPALA-8755: Backend support for Z-ordering .. IMPALA-8755: Backend support for Z-ordering This change depends on gerrit.cloudera.org/#/c/13955/ (Frontend support for Z-ordering) The commit adds a Comperator based on Z-ordering. See in detail: https://en.wikipedia.org/wiki/Z-order_curve The comperator instead of calculating the Z-values of the rows, looks for the column with the most significant dimension, and compares the values of this column only. The most significant dimension will be the one where the compared values have the highest different bits. The algorithm requires values of the same binary representation, but this can be relaxed. Currently, strings, varchars, floats and doubles are not supported. Testing: * Added unit tests. * Currently, some tests are missing. * Run manual tests, comparing 4-column values with 4-bit integers, for all possible combinations. Checked the result by calculating the Z-value for each comparison. * Tested performance on various data, getting great results. Change-Id: I0200748ce3e65ebc5d3530f794c0f80aa335a2ab --- M be/src/exec/exchange-node.cc M be/src/exec/hdfs-table-sink.cc M be/src/exec/hdfs-table-sink.h M be/src/exec/parquet/hdfs-parquet-table-writer.cc M be/src/exec/partial-sort-node.cc M be/src/exec/partial-sort-node.h M be/src/exec/sort-node.cc M be/src/exec/sort-node.h M be/src/exec/topn-node.cc M be/src/runtime/data-stream-test.cc M be/src/runtime/sorter.cc M be/src/runtime/sorter.h M be/src/util/CMakeLists.txt A be/src/util/tuple-row-compare-test.cc M be/src/util/tuple-row-compare.cc M be/src/util/tuple-row-compare.h 16 files changed, 630 insertions(+), 58 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/80/14080/7 -- To view, visit http://gerrit.cloudera.org:8080/14080 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I0200748ce3e65ebc5d3530f794c0f80aa335a2ab Gerrit-Change-Number: 14080 Gerrit-PatchSet: 7 Gerrit-Owner: Norbert Luksa Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-8755: Frontend support for Z-ordering
Norbert Luksa has posted comments on this change. ( http://gerrit.cloudera.org:8080/13955 ) Change subject: IMPALA-8755: Frontend support for Z-ordering .. Patch Set 12: Thank you, Gábor, for the review and the advices. -- To view, visit http://gerrit.cloudera.org:8080/13955 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie122002ca8f52ca2c1e1ec8ff1d476ae1f4f875d Gerrit-Change-Number: 13955 Gerrit-PatchSet: 12 Gerrit-Owner: Norbert Luksa Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 12 Sep 2019 13:37:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8940: Fix MetastoreEventsProcessorTest.testPartitionEvents in CDP builds
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14213 ) Change subject: IMPALA-8940: Fix MetastoreEventsProcessorTest.testPartitionEvents in CDP builds .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/14213 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie93e7237944142f616a80dec675280b397fe5995 Gerrit-Change-Number: 14213 Gerrit-PatchSet: 3 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 12 Sep 2019 13:15:37 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8940: Fix MetastoreEventsProcessorTest.testPartitionEvents in CDP builds
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14213 ) Change subject: IMPALA-8940: Fix MetastoreEventsProcessorTest.testPartitionEvents in CDP builds .. IMPALA-8940: Fix MetastoreEventsProcessorTest.testPartitionEvents in CDP builds The problem was caused by "numFilesErasureCoded" property, which does not exist in CDP builds, and trivial event filtering only skipped events where a "trivial" property was changed, but not when it was added. Change-Id: Ie93e7237944142f616a80dec675280b397fe5995 Reviewed-on: http://gerrit.cloudera.org:8080/14213 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java 1 file changed, 6 insertions(+), 2 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/14213 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ie93e7237944142f616a80dec675280b397fe5995 Gerrit-Change-Number: 14213 Gerrit-PatchSet: 4 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8940: Fix MetastoreEventsProcessorTest.testPartitionEvents in CDP builds
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14213 ) Change subject: IMPALA-8940: Fix MetastoreEventsProcessorTest.testPartitionEvents in CDP builds .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4552/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14213 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie93e7237944142f616a80dec675280b397fe5995 Gerrit-Change-Number: 14213 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 12 Sep 2019 09:44:29 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8940: Fix MetastoreEventsProcessorTest.testPartitionEvents in CDP builds
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14213 ) Change subject: IMPALA-8940: Fix MetastoreEventsProcessorTest.testPartitionEvents in CDP builds .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4942/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/14213 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie93e7237944142f616a80dec675280b397fe5995 Gerrit-Change-Number: 14213 Gerrit-PatchSet: 3 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 12 Sep 2019 09:04:56 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8940: Fix MetastoreEventsProcessorTest.testPartitionEvents in CDP builds
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14213 ) Change subject: IMPALA-8940: Fix MetastoreEventsProcessorTest.testPartitionEvents in CDP builds .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14213 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie93e7237944142f616a80dec675280b397fe5995 Gerrit-Change-Number: 14213 Gerrit-PatchSet: 3 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 12 Sep 2019 09:04:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8940: Fix MetastoreEventsProcessorTest.testPartitionEvents in CDP builds
Csaba Ringhofer has posted comments on this change. (
http://gerrit.cloudera.org:8080/14213 )
Change subject: IMPALA-8940: Fix
MetastoreEventsProcessorTest.testPartitionEvents in CDP builds
..
Patch Set 2: Code-Review+2
(1 comment)
Carry +2
http://gerrit.cloudera.org:8080/#/c/14213/1/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java
File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java:
http://gerrit.cloudera.org:8080/#/c/14213/1/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@1382
PS1, Line 1382: if (val == null) {
> nit: formatting, don't think we use single line if else
Done
--
To view, visit http://gerrit.cloudera.org:8080/14213
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie93e7237944142f616a80dec675280b397fe5995
Gerrit-Change-Number: 14213
Gerrit-PatchSet: 2
Gerrit-Owner: Csaba Ringhofer
Gerrit-Reviewer: Alex Behm
Gerrit-Reviewer: Anurag Mantripragada
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Vihang Karajgaonkar
Gerrit-Comment-Date: Thu, 12 Sep 2019 09:04:00 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8940: Fix MetastoreEventsProcessorTest.testPartitionEvents in CDP builds
Hello Bharath Vissapragada, Anurag Mantripragada, Vihang Karajgaonkar, Alex Behm, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14213 to look at the new patch set (#2). Change subject: IMPALA-8940: Fix MetastoreEventsProcessorTest.testPartitionEvents in CDP builds .. IMPALA-8940: Fix MetastoreEventsProcessorTest.testPartitionEvents in CDP builds The problem was caused by "numFilesErasureCoded" property, which does not exist in CDP builds, and trivial event filtering only skipped events where a "trivial" property was changed, but not when it was added. Change-Id: Ie93e7237944142f616a80dec675280b397fe5995 --- M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java 1 file changed, 6 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/13/14213/2 -- To view, visit http://gerrit.cloudera.org:8080/14213 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie93e7237944142f616a80dec675280b397fe5995 Gerrit-Change-Number: 14213 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
