The advantage of shading/JarJaring our dependencies is we *don't* get stuck with hbase-A.B-hadoop-Z.Y. Those releases are a hassle for users, and a hassle for release managers. The disadvantage being runtime bloat in the form of an excessive number of classes to load.
I think there's a middle ground where we shade only those with a bad track-record. Enis has a good handle on this list. On Wed, Mar 11, 2015 at 3:38 PM, Enis Söztutar <enis....@gmail.com> wrote: > > > > > > Can we do something even more generic like shade [1] our dependencies for > > some of the common sources of dependency pain like guava, jackson, and > > netty? This way we decouple hbase's pain from hadoop's and allow clients > > to choose their own versions of libs to use. We'd likely still cause > some > > inconvenience for 1.0 ->1.1 but this should eliminate this problem from > > biting us again if we are on hadopo 3.x->3.x+1 and hbase 2.y and hbase > > 2.y+1. > > > > I think shading some of the dependencies is a good idea. Obvious stuff are > protobuf, netty, guava, jackson, etc. > We should also convince Hadoop to do it as well. We should decouple our > guava and > protobuf version from that of Hadoop's. I've read that elastic search uses > maven shade > plugin, we can look at what they have done. > > > > > > Jon. > > > > > > [1] http://maven.apache.org/plugins/maven-shade-plugin/ > > > > On Wed, Mar 11, 2015 at 2:04 PM, Enis Söztutar <e...@apache.org> wrote: > > > > > Over at https://issues.apache.org/jira/browse/HBASE-13149, there is > some > > > discussion about changing the jackson version from 1.8 to 1.9 that is > > worth > > > bringing here because of the wider audience. The discussion is about > > > jackson version specific in this issue, but we should also consider > > future > > > changes to libs. > > > > > > The question is, whether we can change the dep version of jackson from > > 1.8 > > > to 1.9 in HBase-1.1. According to our guidelines, we said that we will > > not > > > do breaking changes to our deps in minor versions. From the analysis of > > > jackson compat attached at jira, it seems that jackson have some > changes > > > breaking BC. > > > > > > So, from a purist perspective, if we follow our guidelines literally, > we > > > should not make this change. However, I think we should make that > change > > in > > > 1.1 (and make similar changes in the future as well), because: > > > > > > 1. Hadoop upgraded its version of jackson in 2.5, and HBase MR jobs > fail > > > flat out without corresponding change in HBase (with Hadoop-2.5+). We > do > > > not have any control over Hadoop or similar core dependencies. We > cannot > > be > > > realistically expected to guarantee more than what they do. I would > > rather > > > do the change in 1.1 and not inconvenience the users, than not do the > > > change, and have to explain how to replace the jars in the docs. This > > > argument should be extended beyond the jackson dependency. > > > > > > 2. HBase is not at the maturity level for following the dependency > compat > > > guide literally without reducing the dep footprint, better isolation of > > MR > > > / Mini cluster out of hbase-server and possible participation from > > hadoop. > > > We cannot easily bump up the major version for these as well, since > major > > > version implies other things. > > > > > > So, my proposal is: > > > - Commit HBASE-13149 to master and 1.1 > > > - Either change the dependency compat story for minor versions to > false, > > > or add a footnote saying that there may be exceptions because of the > > > reasons listed above. > > > > > > Let me know what you guys think. > > > Enis > > > > > > > > > > > -- > > // Jonathan Hsieh (shay) > > // HBase Tech Lead, Software Engineer, Cloudera > > // j...@cloudera.com // @jmhsieh > > >
