Always the pragmatist. I accept your proposal with acknowledgement of an effort to improve the state of things going forward.
On Thursday, March 12, 2015, Enis Söztutar <enis....@gmail.com> wrote: > This is good discussion, but I would like to reach a consensus and move on > with HBASE-13149. > > My conclusion from the thread is that we cannot be realistically expected > to keep dependency compat between minor versions because of lack of shading > in HBase and Hadoop, and our dependencies are themselves not semver, and we > cannot promise more than our dependencies promise. > > So I would like to formally drop support for dependency compat between > minor versions as defined in > https://hbase.apache.org/book.html#hbase.versioning. We can reintroduce > later when we have better isolation/guarantees. In the mean time, we can > move on. > > The PMC has approved the compat guide, but I am not sure whether we need a > VOTE thraed. What do you guys think? > > Enis > > On Wed, Mar 11, 2015 at 11:32 PM, Andrew Purtell <apurt...@apache.org > <javascript:;>> > wrote: > > > When did we retire 0.98 (grin) > > > > Seriously, let's not have Hadoop version specific builds beyond 0.98. > Every > > other week or so we find and fix breakage in the Hadoop 1 build because > > nobody develops with Hadoop 1 or runs it in production. The combinatorics > > will be problematic starting with only two branches based on this > > experience. > > > > I don't see how coprocessors can be isolated from our dependencies unless > > we implement classloader/environment support for that, which would be a > lot > > of work and yet not isolate them from other changes they'll face due to > > proximity to other low level details. > > > > > > On Wednesday, March 11, 2015, Sean Busbey <bus...@cloudera.com > <javascript:;>> wrote: > > > > > ugh, yes please let's not return to the days of hadoop-version-specific > > > HBase builds. > > > > > > I'm just spinning up on isolating third-party dependencies over in > > Hadoop. > > > I'd be happy to apply whatever works there within HBase. This only > really > > > helps us for HBase 2.y though, right? > > > > > > Without sidetracking this thread too much, before I file a jira for > > > isolating third-party deps what scope are we looking for? hbase-client > > and > > > the MR tooling only? i.e I'd prefer to state that folks making use of > > > LimitedPrivate features like coprocessors don't get isolation from our > > > dependencies. > > > > > > > > > On Wed, Mar 11, 2015 at 5:44 PM, Nick Dimiduk <ndimi...@gmail.com > <javascript:;> > > > <javascript:;>> wrote: > > > > > > > The advantage of shading/JarJaring our dependencies is we *don't* get > > > stuck > > > > with hbase-A.B-hadoop-Z.Y. Those releases are a hassle for users, > and a > > > > hassle for release managers. The disadvantage being runtime bloat in > > the > > > > form of an excessive number of classes to load. > > > > > > > > I think there's a middle ground where we shade only those with a bad > > > > track-record. Enis has a good handle on this list. > > > > > > > > On Wed, Mar 11, 2015 at 3:38 PM, Enis Söztutar <enis....@gmail.com > <javascript:;> > > > <javascript:;>> wrote: > > > > > > > > > > > > > > > > > > > > > > Can we do something even more generic like shade [1] our > > dependencies > > > > for > > > > > > some of the common sources of dependency pain like guava, > jackson, > > > and > > > > > > netty? This way we decouple hbase's pain from hadoop's and allow > > > > clients > > > > > > to choose their own versions of libs to use. We'd likely still > > cause > > > > > some > > > > > > inconvenience for 1.0 ->1.1 but this should eliminate this > problem > > > from > > > > > > biting us again if we are on hadopo 3.x->3.x+1 and hbase 2.y and > > > hbase > > > > > > 2.y+1. > > > > > > > > > > > > > > > > I think shading some of the dependencies is a good idea. Obvious > > stuff > > > > are > > > > > protobuf, netty, guava, jackson, etc. > > > > > We should also convince Hadoop to do it as well. We should decouple > > our > > > > > guava and > > > > > protobuf version from that of Hadoop's. I've read that elastic > search > > > > uses > > > > > maven shade > > > > > plugin, we can look at what they have done. > > > > > > > > > > > > > > > > > > > > > > Jon. > > > > > > > > > > > > > > > > > > [1] http://maven.apache.org/plugins/maven-shade-plugin/ > > > > > > > > > > > > On Wed, Mar 11, 2015 at 2:04 PM, Enis Söztutar <e...@apache.org > <javascript:;> > > > <javascript:;>> > > > > wrote: > > > > > > > > > > > > > Over at https://issues.apache.org/jira/browse/HBASE-13149, > there > > > is > > > > > some > > > > > > > discussion about changing the jackson version from 1.8 to 1.9 > > that > > > is > > > > > > worth > > > > > > > bringing here because of the wider audience. The discussion is > > > about > > > > > > > jackson version specific in this issue, but we should also > > consider > > > > > > future > > > > > > > changes to libs. > > > > > > > > > > > > > > The question is, whether we can change the dep version of > jackson > > > > from > > > > > > 1.8 > > > > > > > to 1.9 in HBase-1.1. According to our guidelines, we said that > we > > > > will > > > > > > not > > > > > > > do breaking changes to our deps in minor versions. From the > > > analysis > > > > of > > > > > > > jackson compat attached at jira, it seems that jackson have > some > > > > > changes > > > > > > > breaking BC. > > > > > > > > > > > > > > So, from a purist perspective, if we follow our guidelines > > > literally, > > > > > we > > > > > > > should not make this change. However, I think we should make > that > > > > > change > > > > > > in > > > > > > > 1.1 (and make similar changes in the future as well), because: > > > > > > > > > > > > > > 1. Hadoop upgraded its version of jackson in 2.5, and HBase MR > > jobs > > > > > fail > > > > > > > flat out without corresponding change in HBase (with > > Hadoop-2.5+). > > > We > > > > > do > > > > > > > not have any control over Hadoop or similar core dependencies. > We > > > > > cannot > > > > > > be > > > > > > > realistically expected to guarantee more than what they do. I > > would > > > > > > rather > > > > > > > do the change in 1.1 and not inconvenience the users, than not > do > > > the > > > > > > > change, and have to explain how to replace the jars in the > docs. > > > This > > > > > > > argument should be extended beyond the jackson dependency. > > > > > > > > > > > > > > 2. HBase is not at the maturity level for following the > > dependency > > > > > compat > > > > > > > guide literally without reducing the dep footprint, better > > > isolation > > > > of > > > > > > MR > > > > > > > / Mini cluster out of hbase-server and possible participation > > from > > > > > > hadoop. > > > > > > > We cannot easily bump up the major version for these as well, > > since > > > > > major > > > > > > > version implies other things. > > > > > > > > > > > > > > So, my proposal is: > > > > > > > - Commit HBASE-13149 to master and 1.1 > > > > > > > - Either change the dependency compat story for minor versions > > to > > > > > false, > > > > > > > or add a footnote saying that there may be exceptions because > of > > > the > > > > > > > reasons listed above. > > > > > > > > > > > > > > Let me know what you guys think. > > > > > > > Enis > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > // Jonathan Hsieh (shay) > > > > > > // HBase Tech Lead, Software Engineer, Cloudera > > > > > > // j...@cloudera.com <javascript:;> <javascript:;> // @jmhsieh > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Sean > > > > > > > > > -- > > Best regards, > > > > - Andy > > > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > > (via Tom White) > > >