On Wed, Mar 18, 2015 at 7:38 AM, Jonathan Hsieh <[email protected]> wrote:
> We are now considered a mature project and as a "middle turtle" project we > need to consider the projects that we will inconvenience because > they depend on us. All the folks in this discussion are hbase devs and or > sophisticated hbase users. I agree with one of Sean's thoughts -- let us > validate if this is a concern for users or not by have a discussion on > user@ > to see how our users feel about it. Maybe this is no big deal for them. This probably can serve as a head-up for the user community as well. > > Let's say that this is a concern for the users. I like Lars's pragmatic > critieria -- if the dep change doesn't cause problems for intra version > interoperability then, let's let it in and let's update our policy. If it > does cause problem's let's keep it out and stick to our guns. > > To be more concrete there are two scenarios we could test: > 1) an mr job that uses hbase 1.0 client (and it's hadoop dep) needs to > work out of the box against 1.1 hbase servers (and it's hadoop deps) or a > mixed hbase 1.1- hbase 1.0 environment (e.g. assume we're in the middle of > rolling upgrading hbase) without failing due to dependency compat checks. > See the coverage matrix below > 2) an mr job that uses hbase 1.0 client (and it's hadoop dep) needs to work > out of the box against 1.1 hbase servers while the hdfs before and after it > is upgraded (either rolling or shutdown-restart upgrade) > If both succeed despite dep changes, we are in in the clear. If it fails > then we do not allow the dep change. > (hbase 1.0 client w/ jackson 1.8 + hadoop 2.4 ) go against (hbase 1.1 server w/ jackson 1.9 + hadoop 2.4) (hbase 1.0 client w/ jackson 1.8 + hadoop 2.4 ) go against (hbase 1.1 server w/ jackson 1.9 + hadoop 2.6) (hbase 1.0 client w/ jackson 1.8 + hadoop 2.6 ) go against (hbase 1.1 server w/ jackson 1.9 + hadoop 2.6) (hbase 1.0 client w/ jackson 1.8 + hadoop 2.4 ) go against (mixed of hbase 1.0 w/ jackson 1.8 + hbase 1.1 server w/ Jackson 1.9 + hadoop 2.6) Is this the coverage you meant? > To avoid and simplify this problem on our internal builds, we have done the > exercise of harmonizing all to one version of jackson/guava/etc or shading > jars for the combination of hadoop and hbase (and other systems) that we be > ship. I'd like to be in a world where this work is upstream. At the > moment, we don't know whether an upgrade to 2.6 hadoop including its > Jackson 1.9 dep will have MR jobs fail or succeed because of classpath > ordering issues. It sounds like it is an may be an issue but we should > test to make sure and decide based on that. > > Jon. > >
