I'm usually an advocate for getting rid of unnecessary dependencies (cough, jetty, cough), but a lot of the things in Guava are really useful.
Immutable collections, BiMap, Multisets, Arrays#asList, the stuff for writing hashCode() and equals(), String#Joiner, the list goes on. We particularly use the Cache/CacheBuilder stuff a lot in HDFS to get maps with LRU eviction without writing a lot of boilerplate. The QJM stuff uses ListenableFuture a lot, although perhaps we could come up with our own equivalent for that. On Mon, Nov 10, 2014 at 9:26 AM, Alejandro Abdelnur <tuc...@gmail.com> wrote: > IMO we should: > > 1* have a clean and thin client API JAR (which does not drag any 3rd party > dependencies, or a well defined small set -i.e. slf4j & log4j-) > 2* have a client implementation that uses a classloader to isolate client > impl 3rd party deps from app dependencies. > > #2 can be done using a stock URLClassLoader (i would just subclass it to > forbid packages in the API JAR and exposed 3rd parties to be loaded from > the app JAR) > > #1 is the tricky thing as our current API modules don't have a clean > API/impl separation. > > thx > PS: If folks are interested in pursing this, I can put together a prototype > of how #2 would work (I don't think it will be more than 200 lines of code) Absolutely, I agree that we should not be using Guava types in public APIs. Guava has not been very responsible with backwards compatibility, that much is clear. A client / server jar separation is an interesting idea. But then we still have to get rid of Guava and other library deps in the client jars. I think it would be more work than it seems. For example, the HDFS client uses Guava Cache a lot, so we'd have to write our own version of this. Can't we just shade this stuff? Has anyone tried shading Hadoop's Guava? best, Colin > > > On Mon, Nov 10, 2014 at 5:18 AM, Steve Loughran <ste...@hortonworks.com> > wrote: > >> Yes, Guava is a constant pain; there's lots of open JIRAs related to it, as >> its the one we can't seamlessly upgrade. Not unless we do our own fork and >> reinsert the missing classes. >> >> The most common uses in the code are >> >> @VisibleForTesting (easily replicated) >> and the Precondition.check() operations >> >> The latter is also easily swapped out, and we could even add the check they >> forgot: >> Preconditions.checkArgNotNull(argname, arg) >> >> >> These are easy; its the more complex data structures that matter more. >> >> I think for Hadoop 2.7 & java 7 we need to look at this problem and do >> something. Even if we continue to ship Guava 11 so that the HBase team >> don't send any (more) death threats, we can/should rework Hadoop to build >> and run against Guava 16+ too. That's needed to fix some of the recent java >> 7/8+ changes. >> >> -Everything in v11 dropped from v16 MUST to be implemented with our own >> versions. >> -anything tagged as deprecated in 11+ SHOULD be replaced by newer stuff, >> wherever possible. >> >> I think for 2.7+ we should add some new profiles to the POM, for Java 8 and >> 9 alongside the new baseline java 7. For those later versions we could >> perhaps mandate Guava 16. >> >> >> >> On 10 November 2014 00:42, Arun C Murthy <a...@hortonworks.com> wrote: >> >> > … has been a constant pain w.r.t compatibility etc. >> > >> > Should we consider adopting a policy to not use guava in >> Common/HDFS/YARN? >> > >> > MR doesn't matter too much since it's application-side issue, it does >> hurt >> > end-users though since they still might want a newer guava-version, but >> at >> > least they can modify MR. >> > >> > Thoughts? >> > >> > thanks, >> > Arun >> > >> > >> > -- >> > CONFIDENTIALITY NOTICE >> > NOTICE: This message is intended for the use of the individual or entity >> to >> > which it is addressed and may contain information that is confidential, >> > privileged and exempt from disclosure under applicable law. If the reader >> > of this message is not the intended recipient, you are hereby notified >> that >> > any printing, copying, dissemination, distribution, disclosure or >> > forwarding of this communication is strictly prohibited. If you have >> > received this communication in error, please contact the sender >> immediately >> > and delete it from your system. Thank You. >> > >> >> -- >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity to >> which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender immediately >> and delete it from your system. Thank You. >>
