Tim: Sounds good! Grant: Thanks for the broad testing! Let me know if you think we need to additionally document anything. Andrew: Thanks for testing on High Sierra and for being accommodating on this!
Mike On Tue, Mar 5, 2019 at 10:26 AM Andrew Wong <[email protected]> wrote: > I ran your example on macOS High Sierra 10.13.6 and on CentOS 7.3, only > issue I ran into was because of the Maven version. > I'll let this sit for the day or so in case people want to chyme in, and > then proceed with the release. > > Thanks for pushing this through! > > On Tue, Mar 5, 2019 at 8:44 AM Grant Henke <[email protected]> > wrote: > > > +1 on this plan. Thanks for wrapping this up Mike! > > > > I ran the test locally on my Mac (10.14.2) with no issues. > > > > I also ran the tests in a docker image for the following operating > systems: > > > > - centos:6 > > - centos:7 > > - debian:jessie > > - debian:stretch > > - ubuntu:trusty > > - ubuntu:xenial > > - ubuntu:bionic > > > > I ran into a few hiccups during testing, but they were all easy to > resolve: > > > > - I needed to upgrade the default Maven version on ubuntu:trusty and > > centos. > > - I ran into a open-jdk/maven-surefire bug on debian:stretch > > - > > > > > https://stackoverflow.com/questions/53010200/maven-surefire-could-not-find-forkedbooter-class > > - Fixed with: > > > > > > > > > > > > > > > > > > > > > > *<plugin> <groupId>org.apache.maven.plugins</groupId> > > > <artifactId>maven-surefire-plugin</artifactId> <configuration> > > > <useSystemClassLoader>false</useSystemClassLoader> > > > </configuration></plugin>* > > > > > > The setup varies a bit based on the OS but my basic test script was > > effectively this: > > > > *docker run -it --rm --privileged ubuntu:xenial /bin/bash* > > > > > > > *apt-get update -y* > > > > *apt-get install -y git maven ntp openjdk-8-jdk* > > > > > > > *echo "listen on *" >> /etc/ntp.conf* > > > > *service ntp restart* > > > > > > > *git clone https://github.com/mpercy/kudu-binary-jar-test/ > > > <https://github.com/mpercy/kudu-binary-jar-test/>* > > > > *cd kudu-binary-jar-test* > > > > *mvn test* > > > > > > > > > > > > On Tue, Mar 5, 2019 at 2:57 AM Tim Robertson <[email protected]> > > wrote: > > > > > Thank you for progressing all this Mike (and others). > > > I will endeavour to test this soon and replace the Beam unit tests with > > > this - I've been sidelined a bit recently with other tasks but getting > > back > > > into Beam development again. > > > > > > > > > > > > > > > On Tue, Mar 5, 2019 at 9:16 AM Mike Percy <[email protected]> wrote: > > > > > > > Hi devs, > > > > For the 1.9.0 release, we (Grant Henke, Brian McDevitt, and I) > > > implemented > > > > the capability to start a Kudu minicluster from a JVM environment > > without > > > > having to first build Kudu locally (work tracked as KUDU-2411 > > > > <https://issues.apache.org/jira/browse/KUDU-2411>). This was made > > > possible > > > > by uploading relocatable Kudu binaries (Linux and Mac) to Maven > > central, > > > > exposing the Kudu MiniCluster libraries for use by external projects, > > and > > > > adding logic to the MiniCluster to search the classpath for the Kudu > > > > binaries. > > > > > > > > The completion of this work landed a little late in the 1.9.0 release > > > > cycle, just before we started creating release candidates, and as a > > > result > > > > the binaries didn't make it into the RC process for a vote. I propose > > we > > > > release binary convenience artifacts based on the source tag without > > > going > > > > through another vote because (1) they were built with the 1.9.0 RC2 > > > source > > > > code as voted upon, (2) they are provided simply for convenience and > > are > > > > not needed to use Kudu, and (3) they are currently considered > > > experimental. > > > > > > > > I have already audited the binary artifacts for licensing issues and > > put > > > in > > > > place automation > > > > < > > > > > > > > > > https://github.com/apache/kudu/blob/1.9.0-RC2/build-support/mini-cluster/check-license.pl > > > > > > > > > for > > > > the license checks at artifact build time. README, NOTICE, and > LICENSE > > > are > > > > all included. > > > > > > > > If you're interested in these artifacts and would like to see if they > > > work > > > > for you, or would like to take a closer look for other reasons, I've > > > > uploaded binary jar artifacts for Linux and Mac (h/t to Alexey for > the > > El > > > > Capitan build) to the Maven staging repository at > > > > > https://repository.apache.org/content/repositories/orgapachekudu-1033/ > > > > > > > > We also have an example of using the above binary artifacts, built > from > > > the > > > > 1.9.0-RC2 tag and using 1.9.0-RC2 Java client libraries at > > > > https://github.com/mpercy/kudu-binary-jar-test/ -- you can see if > the > > > > artifact works in your environment by simply cloning that Git > > repository > > > > and running "mvn test". (This is just a copy/pasted and modified > > version > > > of > > > > java-example from the Kudu code examples repo; h/t to Brian for > getting > > > it > > > > to work with the binary jar.) > > > > > > > > Please note that the binary jar Maven integration (using > > os-maven-plugin) > > > > requires Maven 3.1 or later. In addition, it must be run on macOS El > > > > Capitan or later, or a modern flavor of Linux (any Linux distribution > > > that > > > > Kudu supports > > > > < > > > > > > > > > > https://kudu.apache.org/docs/installation.html#_prerequisites_and_requirements > > > > > > > > > should work). > > > > > > > > A draft of user docs for how to integrate the Kudu binary jar into a > > Java > > > > project can be found here > > > > < > > > > > > > > > > https://docs.google.com/document/d/1n9flFdHEnULX3_wxmy__gU7WKK-FIYFxvufWbvEHu-c/ > > > > > > > > > and > > > > if we have consensus on including this as outlined above then I'll > work > > > > with Andrew to get the relevant docs posted to the web site in time > for > > > the > > > > 1.9.0 release announcement. > > > > > > > > Thanks to those who helped get this done (some early prototyping was > > also > > > > done by Tim Robertson) and I hope that folks find this new capability > > > > useful! > > > > > > > > Please let me know if you have questions, concerns, or feedback on > this > > > > functionality. > > > > > > > > Regards, > > > > Mike > > > > > > > > > > > > > -- > > Grant Henke > > Software Engineer | Cloudera > > [email protected] | twitter.com/gchenke | linkedin.com/in/granthenke > > >
