Thanks Attila That’s great feedback and helpful for me to reference as guidance.
By “Kudu installation” I was referring to the possibility that an install might set config etc, beyond just having the binary. I got it running on CentOS similar to how you outline now. I too believe mocking makes most sense, especially as we have the IT running as well, but was asked to explore this further. It’s useful to know you’d agree. Thanks Tim > On 29 Jun 2018, at 17:33, Attila Bukor <abu...@cloudera.com> wrote: > > Hi Tim, > > I’m not sure what you mean by relying on actual installations. If you have > the kudu, kudu-master and kudu-tserver binaries at the same location and they > can be executed, MiniKuduCluster can be used (“binDir” property should be set > to the directory containing the Kudu binaries). You should also look into > BaseKuduTest as that will set up the MiniKuduCluster for you and you don’t > have to do it manually. > > Extracting the Kudu binaries from an rpm should probably work, but that binds > you to CDH as currently Cloudera is the only one that ships Kudu binaries and > MacOS builds are not available anywhere afaik. Also, 1.4.0 is around a year > old, you might want to use this repository instead (from CDH 5.13 Kudu is > part of the CDH): > http://archive.cloudera.com/cdh5/redhat/7/x86_64/cdh/5/RPMS/x86_64/kudu-1.7.0+cdh5.15.0+0-1.cdh5.15.0.p0.52.el7.x86_64.rpm > > As a general suggestion, I would recommend mocking Kudu for unit tests > (that’s what a unit test is for after all) and create separate integration > tests that actually use Kudu that can be skipped where Kudu is not available. > Of course the CI should be set up to be able to provide all necessary > integrations for the tests, but a developer wouldn’t have to set up Kudu, or > use Docker to run the tests if their change doesn’t affect the Kudu > integration. > > Attila > >> On 2018. Jun 29., at 16:42, Tim Robertson <timrobertson...@gmail.com> wrote: >> >> Hi folks, >> >> I've written Java KuduIO for Apache Beam with integration tests making use >> of Kudu in Docker. It is yet to be committed on Apache Beam. >> >> Rather than mocking Kudu client for unit tests I'd like to explore use of >> the MiniKuduCluster which "Depends on precompiled kudu, kudu-master, and >> kudu-tserver binaries". >> >> I'd need unit tests to run on the main linux distros and OS X. >> >> For the linux distros, would an approach where I extract the binaries from >> the packages [1] work please? Or does the MiniKuduCluster rely on actual >> installations? I am pretty weak on C builds and linked libraries etc (Java >> guy, sorry). >> >> For CentOS I'm exploring this for example: >> rpm2cpio ./kudu-1.4.0+cdh5.12.2+0-1.cdh5.12.2.p0.8.el7.x86_64.rpm | cpio >> -idmv >> >> I haven't explored OS X options yet. >> >> Any advice here would greatly be appreciated to save me going down a dead >> end. >> >> Many thanks, >> Tim >> >> >> [1] http://kudu.apache.org/docs/installation.html#install_packages > >
