Adar Dembo has posted comments on this change. Change subject: WIP: use C++ ExternalMiniCluster for Java and Python tests ......................................................................
Patch Set 2: (2 comments) > I think it's a great idea. What about more machine-oriented > interface for the CLI tool? Do you expect it to be transformed > into something JSON-like in the nearest future? See the comment Todd left; we're discussing just that. > Maybe, it's worth introducing running a proxy along with > minicluster and providing something like REST interface instead of > CLI for the tests? I didn't implement a TCP-based connection between the proxy and the tests exactly so that the entire "port already in use" class of issues can be avoided. When communication is over a TCP socket, we either need to use a well-known port, which is prone to conflicts, or an ephemeral port, whose number needs to be communicated back to the tests. If we've already got a channel for communicating the port number (probably stdout), we can use that channel for control too. I think a socket-based approach would make more sense if there were multiple consumers of a single mini cluster, but that's just not the case with our tests. http://gerrit.cloudera.org:8080/#/c/7853/2//COMMIT_MSG Commit Message: PS2, Line 20: authz > nit here and below: I think it should be 'authn' -- the kerberos-related ac You are right, my bad. PS2, Line 47: WIP because, well, it should be pretty obvious. I was able to get through a : full run of "mvn verify" locally, so I have confidence that this can work. : But I'd like to solicit feedback on the general approach before spending : more time applying spit and polish. > general approach seems reasonable to me. I went back and forth on this. JSON (or protobuf, or thrift, or or or...) would certainly make the RPC system far more robust and maintainable. But we'd lose the ability to actually use run_cluster interactively from the command line, because no one wants to write JSON or whatever by hand. Right now the noun/verb word-based RPC allows for interactivity and that's how I did much of the early testing. It's not robust, but it won't be broken by simple things like e.g. a minicluster dir with a space (it would be broken by unexpected newlines though). Do you think the pros of using a real serialization format outweigh the loss of interactivity? -- To view, visit http://gerrit.cloudera.org:8080/7853 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I0e693921ef780dc4a06e536c6b7408f7f0b252f6 Gerrit-PatchSet: 2 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Adar Dembo <a...@cloudera.com> Gerrit-Reviewer: Adar Dembo <a...@cloudera.com> Gerrit-Reviewer: Alexey Serbin <aser...@cloudera.com> Gerrit-Reviewer: Dan Burkert <danburk...@apache.org> Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tidy Bot Gerrit-Reviewer: Todd Lipcon <t...@apache.org> Gerrit-HasComments: Yes
