+David (with whom we just spoke about this very subject):
On Fri, Jun 8, 2012 at 2:54 PM, Andrew Purtell <[email protected]> wrote:
> This is a HBase shell nit specifically, and the warning from SLF4J is
> purely a warning:
>
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/lib/hadoop/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/lib/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for
> an explanation.
>
> but there are other cases where dependencies in common to more than
> one project will appear on the classpath, sometimes with differing
> versions. Jackson comes to mind.
This is very much a problem for Bigtop (and frankly any kind of Hadoop
distribution on the market) although it hasn't caused us much trouble
yet. Still, I really think we should address this. Here's a JIRA that
tracks the effort:
https://issues.apache.org/jira/browse/BIGTOP-276
this is clearly post Bigtop 0.4.0 but it could very well make it into
Bigtop 0.5.0.
Of course, this is half of the problem -- the other half is making sure
that we can harmonize the versions of all the dependencies between
the projects. We do that in CDH (via patching poms, etc) but haven't
done anything like that in Bigtop yet.
> How this would be handled at the OS level is each common dependency
> would be factored out into a library package.
I'm keeping an eye on what Linux vendors do to address this issue. Here's
a blog post that is well worth a read. I think whatever we end up implementing
in Bigtop needs to be aligned (to the best possible extent) with Linux vendors:
http://duncan.mac-vicar.com/2012/01/26/on-java-maven-jpp-and-rpm/
http://fedoraproject.org/wiki/Packaging:Java#build-classpath
Of course, if anybody has any concrete proposal I'd love for them to
be articulated on the JIRA or mailing list -- do let me know!
Thanks,
Roman.
