I don't think we need a vote unless we cannot reach consensus. I would be in favor of this change, for the stated reason:
> we cannot be realistically expected > to keep dependency compat between minor versions because of lack of > shading in HBase and Hadoop, and our dependencies are themselves > not semver, and we cannot promise more than our dependencies promise. I would also be in favor of revising this later and supportive of efforts to improve dependency isolation if contributors step up to do the work. On Thu, Mar 12, 2015 at 11:20 AM, Enis Söztutar <enis....@gmail.com> wrote: > This is good discussion, but I would like to reach a consensus and move on > with HBASE-13149. > > My conclusion from the thread is that we cannot be realistically expected > to keep dependency compat between minor versions because of lack of shading > in HBase and Hadoop, and our dependencies are themselves not semver, and we > cannot promise more than our dependencies promise. > > So I would like to formally drop support for dependency compat between > minor versions as defined in > https://hbase.apache.org/book.html#hbase.versioning. We can reintroduce > later when we have better isolation/guarantees. In the mean time, we can > move on. > > The PMC has approved the compat guide, but I am not sure whether we need a > VOTE thraed. What do you guys think? > > Enis > > On Wed, Mar 11, 2015 at 11:32 PM, Andrew Purtell <apurt...@apache.org> > wrote: > > > When did we retire 0.98 (grin) > > > > Seriously, let's not have Hadoop version specific builds beyond 0.98. > Every > > other week or so we find and fix breakage in the Hadoop 1 build because > > nobody develops with Hadoop 1 or runs it in production. The combinatorics > > will be problematic starting with only two branches based on this > > experience. > > > > I don't see how coprocessors can be isolated from our dependencies unless > > we implement classloader/environment support for that, which would be a > lot > > of work and yet not isolate them from other changes they'll face due to > > proximity to other low level details. > > > > > > On Wednesday, March 11, 2015, Sean Busbey <bus...@cloudera.com> wrote: > > > > > ugh, yes please let's not return to the days of hadoop-version-specific > > > HBase builds. > > > > > > I'm just spinning up on isolating third-party dependencies over in > > Hadoop. > > > I'd be happy to apply whatever works there within HBase. This only > really > > > helps us for HBase 2.y though, right? > > > > > > Without sidetracking this thread too much, before I file a jira for > > > isolating third-party deps what scope are we looking for? hbase-client > > and > > > the MR tooling only? i.e I'd prefer to state that folks making use of > > > LimitedPrivate features like coprocessors don't get isolation from our > > > dependencies. > > > > > > > > > On Wed, Mar 11, 2015 at 5:44 PM, Nick Dimiduk <ndimi...@gmail.com > > > <javascript:;>> wrote: > > > > > > > The advantage of shading/JarJaring our dependencies is we *don't* get > > > stuck > > > > with hbase-A.B-hadoop-Z.Y. Those releases are a hassle for users, > and a > > > > hassle for release managers. The disadvantage being runtime bloat in > > the > > > > form of an excessive number of classes to load. > > > > > > > > I think there's a middle ground where we shade only those with a bad > > > > track-record. Enis has a good handle on this list. > > > > > > > > On Wed, Mar 11, 2015 at 3:38 PM, Enis Söztutar <enis....@gmail.com > > > <javascript:;>> wrote: > > > > > > > > > > > > > > > > > > > > > > Can we do something even more generic like shade [1] our > > dependencies > > > > for > > > > > > some of the common sources of dependency pain like guava, > jackson, > > > and > > > > > > netty? This way we decouple hbase's pain from hadoop's and allow > > > > clients > > > > > > to choose their own versions of libs to use. We'd likely still > > cause > > > > > some > > > > > > inconvenience for 1.0 ->1.1 but this should eliminate this > problem > > > from > > > > > > biting us again if we are on hadopo 3.x->3.x+1 and hbase 2.y and > > > hbase > > > > > > 2.y+1. > > > > > > > > > > > > > > > > I think shading some of the dependencies is a good idea. Obvious > > stuff > > > > are > > > > > protobuf, netty, guava, jackson, etc. > > > > > We should also convince Hadoop to do it as well. We should decouple > > our > > > > > guava and > > > > > protobuf version from that of Hadoop's. I've read that elastic > search > > > > uses > > > > > maven shade > > > > > plugin, we can look at what they have done. > > > > > > > > > > > > > > > > > > > > > > Jon. > > > > > > > > > > > > > > > > > > [1] http://maven.apache.org/plugins/maven-shade-plugin/ > > > > > > > > > > > > On Wed, Mar 11, 2015 at 2:04 PM, Enis Söztutar <e...@apache.org > > > <javascript:;>> > > > > wrote: > > > > > > > > > > > > > Over at https://issues.apache.org/jira/browse/HBASE-13149, > there > > > is > > > > > some > > > > > > > discussion about changing the jackson version from 1.8 to 1.9 > > that > > > is > > > > > > worth > > > > > > > bringing here because of the wider audience. The discussion is > > > about > > > > > > > jackson version specific in this issue, but we should also > > consider > > > > > > future > > > > > > > changes to libs. > > > > > > > > > > > > > > The question is, whether we can change the dep version of > jackson > > > > from > > > > > > 1.8 > > > > > > > to 1.9 in HBase-1.1. According to our guidelines, we said that > we > > > > will > > > > > > not > > > > > > > do breaking changes to our deps in minor versions. From the > > > analysis > > > > of > > > > > > > jackson compat attached at jira, it seems that jackson have > some > > > > > changes > > > > > > > breaking BC. > > > > > > > > > > > > > > So, from a purist perspective, if we follow our guidelines > > > literally, > > > > > we > > > > > > > should not make this change. However, I think we should make > that > > > > > change > > > > > > in > > > > > > > 1.1 (and make similar changes in the future as well), because: > > > > > > > > > > > > > > 1. Hadoop upgraded its version of jackson in 2.5, and HBase MR > > jobs > > > > > fail > > > > > > > flat out without corresponding change in HBase (with > > Hadoop-2.5+). > > > We > > > > > do > > > > > > > not have any control over Hadoop or similar core dependencies. > We > > > > > cannot > > > > > > be > > > > > > > realistically expected to guarantee more than what they do. I > > would > > > > > > rather > > > > > > > do the change in 1.1 and not inconvenience the users, than not > do > > > the > > > > > > > change, and have to explain how to replace the jars in the > docs. > > > This > > > > > > > argument should be extended beyond the jackson dependency. > > > > > > > > > > > > > > 2. HBase is not at the maturity level for following the > > dependency > > > > > compat > > > > > > > guide literally without reducing the dep footprint, better > > > isolation > > > > of > > > > > > MR > > > > > > > / Mini cluster out of hbase-server and possible participation > > from > > > > > > hadoop. > > > > > > > We cannot easily bump up the major version for these as well, > > since > > > > > major > > > > > > > version implies other things. > > > > > > > > > > > > > > So, my proposal is: > > > > > > > - Commit HBASE-13149 to master and 1.1 > > > > > > > - Either change the dependency compat story for minor versions > > to > > > > > false, > > > > > > > or add a footnote saying that there may be exceptions because > of > > > the > > > > > > > reasons listed above. > > > > > > > > > > > > > > Let me know what you guys think. > > > > > > > Enis > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > // Jonathan Hsieh (shay) > > > > > > // HBase Tech Lead, Software Engineer, Cloudera > > > > > > // j...@cloudera.com <javascript:;> // @jmhsieh > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Sean > > > > > > > > > -- > > Best regards, > > > > - Andy > > > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > > (via Tom White) > > > -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
