Hello Attila Bukor, Grant Henke, Hao Hao,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/15574
to review the following change.
Change subject: subprocess: use a pipe instead of stdout for IO
......................................................................
subprocess: use a pipe instead of stdout for IO
Today, communication from a Kudu Java subprocess to the C++
SubprocessServer is done via the subprocess's stdout (the C++ message
protocol listens on the stdout FD of the subprocess). This is risky
because any amount of logging by the subprocess to stdout may end up
breaking the message protocol, preventing any calls to the subprocess
from succeeding.
This updates the Kudu subprocess module on both the Java and C++ side to
use a named pipe for output instead of the subprocess's stdout. For
Ranger, this pipe will by default be placed in the Ranger config
directory, which, in some distributions (e.g. CDPD) will be unique per
Kudu process.
It also updates the subprocess to share stdout and stderr with the
parent process by default. While this patch doesn't add additional
logging on the Java side, I tested this by adding some log statements to
the Java subprocess and saw them output int he gtest output.
I considered using Netty to encapsulate the unnamed FDs from our pipe()
calls in a Netty FileDescriptor[1], which has useful write() calls. I
opted not to since that brough in some hairiness around its usage of JNI
(I saw an UnsatisfiedLinkError when I tried writing to the FD).
[1] https://netty.io/4.0/api/io/netty/channel/unix/FileDescriptor.html
Change-Id: I63e91a090fe196713a013d28301c7980e452456c
---
M
java/kudu-subprocess/src/main/java/org/apache/kudu/subprocess/SubprocessConfiguration.java
M
java/kudu-subprocess/src/main/java/org/apache/kudu/subprocess/SubprocessExecutor.java
M src/kudu/master/catalog_manager.cc
M src/kudu/master/ranger_authz_provider.cc
M src/kudu/master/ranger_authz_provider.h
M src/kudu/ranger/ranger_client-test.cc
M src/kudu/ranger/ranger_client.cc
M src/kudu/ranger/ranger_client.h
M src/kudu/subprocess/server.cc
M src/kudu/subprocess/server.h
M src/kudu/subprocess/subprocess_proxy-test.cc
M src/kudu/subprocess/subprocess_proxy.h
M src/kudu/subprocess/subprocess_server-test.cc
13 files changed, 173 insertions(+), 27 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/74/15574/1
--
To view, visit http://gerrit.cloudera.org:8080/15574
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I63e91a090fe196713a013d28301c7980e452456c
Gerrit-Change-Number: 15574
Gerrit-PatchSet: 1
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <abu...@apache.org>
Gerrit-Reviewer: Grant Henke <granthe...@apache.org>
Gerrit-Reviewer: Hao Hao <hao....@cloudera.com>
