Hello Tidy Bot, Alexey Serbin, Kudu Jenkins, Andrew Wong, Adar Dembo, Grant
Henke, Hao Hao,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15483
to look at the new patch set (#49).
Change subject: KUDU-3079 Add MiniRanger
......................................................................
KUDU-3079 Add MiniRanger
Apache Ranger has some setup scripts[1][2] that should be used to
install it but they touch a lot of things on the file system that
unfortunately can't be configured or $PREFIX-ed/chroot-ed, so we've
opted for a more manual approach similar to what Impala have chosen[3].
As Ranger 2.0.0 was released back in August 2019 which didn't contain
the Kudu service definition and it wasn't a clean backport I opted to
use the tip of the master (at the time of writing).
Unfortunately Ranger's scripts were written for Python 2 and there were
several compatibility issues with Python 3 so I had to patch it. Most of
the changes were generated by Python's "2to3" script but it required
some manual changes as well to run on Python 3 (subprocesses return
bytes and they were compared with strings which wasn't a problem in
Python 2 but fails in Python 3 so the subprocess result had to be
decoded).
I had to apply another patch to fix an issue running db_setup.py on Mac
as it runs ranger-admin-services.sh which sets the script path by
running `readlink -f $0` which doesn't work on Mac. Instead of rewriting
the script in a way that would work properly on Mac and Linux I opted to
use an environment variable instead, so I decided to use a separate
patch file for this as this one will need to be applied even after
Ranger decides to support Python 3[4].
The commit also adds support for HTTP Basic auth in EasyCurl along with
specifying headers in POST requests which is used in MiniRanger to talk
to the Ranger admin REST API when creating the Kudu service after
startup and adding policies.
Also adds a CreateSymLink() method to Env.
Starting MiniRanger (fresh install) takes ~20s on my machine (2019
MacBook Pro) and ~30-40s on dist-test server.
Starting Ranger subprocess is not handled in this commit, it is going to
be done in a follow-up patch.
Thanks to Andrew Wong for his help with the Ranger setup[5].
[1] https://github.com/apache/ranger/blob/master/security-admin/scripts/setup.sh
[2]
https://github.com/apache/ranger/blob/master/security-admin/scripts/set_globals.sh
[3]
https://github.com/apache/impala/commit/0cb7187841780cabe368607ff559e493be59db22
[4] https://issues.apache.org/jira/browse/RANGER-2759
[5] https://gerrit.cloudera.org/c/15385/5
Change-Id: I15ab1eb8abe71c074c26b286073442882e101bc6
---
M build-support/dist_test.py
M build-support/run_dist_test.py
M src/kudu/mini-cluster/CMakeLists.txt
M src/kudu/mini-cluster/external_mini_cluster.cc
M src/kudu/mini-cluster/external_mini_cluster.h
M src/kudu/postgres/mini_postgres.cc
M src/kudu/ranger/CMakeLists.txt
A src/kudu/ranger/mini_ranger-test.cc
A src/kudu/ranger/mini_ranger.cc
A src/kudu/ranger/mini_ranger.h
A src/kudu/ranger/mini_ranger_configs.h
M src/kudu/security/test/test_pass.h
M src/kudu/server/webserver-test.cc
M src/kudu/tools/tool.proto
M src/kudu/tools/tool_action.h
M src/kudu/tools/tool_action_test.cc
M src/kudu/util/curl_util.cc
M src/kudu/util/curl_util.h
M src/kudu/util/env-test.cc
M src/kudu/util/env.h
M src/kudu/util/env_posix.cc
M thirdparty/LICENSE.txt
M thirdparty/build-thirdparty.sh
M thirdparty/download-thirdparty.sh
A thirdparty/patches/ranger-fixscripts.patch
A thirdparty/patches/ranger-python3.patch
M thirdparty/vars.sh
27 files changed, 1,776 insertions(+), 31 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/83/15483/49
--
To view, visit http://gerrit.cloudera.org:8080/15483
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I15ab1eb8abe71c074c26b286073442882e101bc6
Gerrit-Change-Number: 15483
Gerrit-PatchSet: 49
Gerrit-Owner: Attila Bukor <abu...@apache.org>
Gerrit-Reviewer: Adar Dembo <a...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <aser...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <abu...@apache.org>
Gerrit-Reviewer: Grant Henke <granthe...@apache.org>
Gerrit-Reviewer: Hao Hao <hao....@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
