IMO we should: 1* have a clean and thin client API JAR (which does not drag any 3rd party dependencies, or a well defined small set -i.e. slf4j & log4j-) 2* have a client implementation that uses a classloader to isolate client impl 3rd party deps from app dependencies.
#2 can be done using a stock URLClassLoader (i would just subclass it to forbid packages in the API JAR and exposed 3rd parties to be loaded from the app JAR) #1 is the tricky thing as our current API modules don't have a clean API/impl separation. thx PS: If folks are interested in pursing this, I can put together a prototype of how #2 would work (I don't think it will be more than 200 lines of code) On Mon, Nov 10, 2014 at 5:18 AM, Steve Loughran <ste...@hortonworks.com> wrote: > Yes, Guava is a constant pain; there's lots of open JIRAs related to it, as > its the one we can't seamlessly upgrade. Not unless we do our own fork and > reinsert the missing classes. > > The most common uses in the code are > > @VisibleForTesting (easily replicated) > and the Precondition.check() operations > > The latter is also easily swapped out, and we could even add the check they > forgot: > Preconditions.checkArgNotNull(argname, arg) > > > These are easy; its the more complex data structures that matter more. > > I think for Hadoop 2.7 & java 7 we need to look at this problem and do > something. Even if we continue to ship Guava 11 so that the HBase team > don't send any (more) death threats, we can/should rework Hadoop to build > and run against Guava 16+ too. That's needed to fix some of the recent java > 7/8+ changes. > > -Everything in v11 dropped from v16 MUST to be implemented with our own > versions. > -anything tagged as deprecated in 11+ SHOULD be replaced by newer stuff, > wherever possible. > > I think for 2.7+ we should add some new profiles to the POM, for Java 8 and > 9 alongside the new baseline java 7. For those later versions we could > perhaps mandate Guava 16. > > > > On 10 November 2014 00:42, Arun C Murthy <a...@hortonworks.com> wrote: > > > … has been a constant pain w.r.t compatibility etc. > > > > Should we consider adopting a policy to not use guava in > Common/HDFS/YARN? > > > > MR doesn't matter too much since it's application-side issue, it does > hurt > > end-users though since they still might want a newer guava-version, but > at > > least they can modify MR. > > > > Thoughts? > > > > thanks, > > Arun > > > > > > -- > > CONFIDENTIALITY NOTICE > > NOTICE: This message is intended for the use of the individual or entity > to > > which it is addressed and may contain information that is confidential, > > privileged and exempt from disclosure under applicable law. If the reader > > of this message is not the intended recipient, you are hereby notified > that > > any printing, copying, dissemination, distribution, disclosure or > > forwarding of this communication is strictly prohibited. If you have > > received this communication in error, please contact the sender > immediately > > and delete it from your system. Thank You. > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. >
