So I was able to get a proof-of-concept working [1], although the script is
a bit hacky. The hacky part is that I arbitrarily chose a few system
modules not to package until the thing was willing to run. I was able to
build Kudu on EL6 and run it on Ubuntu 16.04. I have not make it work on
macOS yet... the commands for the rpath modifications to make it
relocatable would be different but the effect should be similar: library
paths relative to the binary.
To get this proof-of-concept to a usable state, we still would need the
following pieces:
1) Make the above script also work on macOS using install_name_tool
and @loader_path per [2]
2) Write a script to upload the resulting tarball to the right place on
artifactory
3) Come up with a way to download the correct binary tarball and unpack it
locally as part of the Maven test phase in dependent projects -- I think we
can use ${os.detected.classifier} from
https://github.com/trustin/os-maven-plugin for the system matching, I
wonder if we need a custom Maven plugin to do the unpacking? Or maybe
somehow we could use the ant-run plugin?
4) Move the KuduMiniCluster out of the kudu-client test dir and into its
own Maven module so it's consumable by other projects
I'm pretty busy this week and would be happy to get some help on this if
anybody has cycles to spare.
Mike
[1] https://gist.github.com/mpercy/aee8d95c55f713615b90df5faad4bb99
[2] https://blogs.oracle.com/dipol/dynamic-libraries,-rpath,-and-mac-os
On Mon, Jul 9, 2018 at 11:36 AM Tim Robertson <timrobertson...@gmail.com>
wrote:
> Slightly related...
>
> Such a build could in theory be easily turned into a docker image too using
> the relatively new Jib tool from Google:
>
>
> https://cloudplatform.googleblog.com/2018/07/introducing-jib-build-java-docker-images-better.html
>
>
>
>
> On Fri, Jul 6, 2018 at 12:57 AM, Mike Percy <mpe...@apache.org> wrote:
>
> > On Thu, Jul 5, 2018 at 1:34 PM Todd Lipcon <t...@cloudera.com.invalid>
> > wrote:
> >
> > > In many cases, builds done on earlier systems are runnable on newer
> > > systems. For example, el6 builds tend to run fine on el7 and ubuntu 14
> in
> > > my experience. If you also bundle the libssl and libcrypto, it seems
> that
> > > an el6 build can also run on Ubuntu 16. So, assuming we could package
> > more
> > > than one binary in the artifact, packaging the shared libraries and
> > setting
> > > RPATH or LD_LIBRARY_PATH accordingly is probably a reasonable option to
> > run
> > > across most common Linux variants.
> >
> >
> > Cool idea. We could do a fully dynamic build, ship all the dynamic deps
> > with hacked rpaths, and likely save some space in the tarball along the
> > way. It's worth trying out.
> >
> > Mike
> >
>
