I believe this fixed some HBase MR test issues when run against 0.23 https://issues.apache.org/jira/browse/HBASE-5212 (but is incompatible with 1.0.0)
FYI, There were some semantics changes in Hadoop 0.23.0 that broke some of the tests with HBase 0.92.0 on it. Here's jiras for those interested. Some other HBase 0.92 test failures related fixed in Hadoop 0.23.x. https://issues.apache.org/jira/browse/HDFS-2838 https://issues.apache.org/jira/browse/HADOOP-7997 Jon On Fri, Jan 27, 2012 at 11:50 AM, Andrew Purtell <[email protected]>wrote: > Ted, > > If I recall correctly, we did consider it. You were welcome at the time to > make this comment on the appropriate JIRA. Maybe that would have changed > the decision. > > Shims are ugly hacks as a rule. > > We also may have overreached trying to be compatible with CDH3, ASF 1.0, > and 0.22, and 0.23. I know the MR tests are broken against 0.23. Also a > change for 1.0 broke against 0.23 if I recall correctly. If I want to get > the tests working against 0.23 there's some work to do. Which may break > against 1.0, requiring more work, which may break against ... > > And what about subtle changes that don't break compilation but require a > combinatorial explosion of test configurations to identify at runtime? > > Where does that end? > > My prediction: A decision to support one version of ZooKeeper, and core, > with everything else YMMV. > > Best regards, > > - Andy > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > (via Tom White) > > > ----- Original Message ----- > > From: Ted Yu <[email protected]> > > To: [email protected]; Andrew Purtell <[email protected]> > > Cc: > > Sent: Friday, January 27, 2012 10:26 AM > > Subject: Re: rethinking zookeeper version > > > > I agree with what Andy and Gary said. > > > > In the future when we encounter incompatible API changes in a very > > important Apache project which HBase depends heavily, we do need to > > consider providing shim so that we have some room to accommodate > different > > releases of the underlying project in the (short) period after new > release > > of HBase. > > > > Cheers > > > > On Fri, Jan 27, 2012 at 10:14 AM, Andrew Purtell > > <[email protected]>wrote: > > > >> If you dig in on ZOOKEEPER-1367, this is a custom ZooKeeper embedding > in a > >> product, not a deployment scenario that one would see with HBase. I > > don't > >> want to overestimate or underestimate the importance of the issue. > >> Currently it is under investigation and the ZK folks haven't gotten to > > the > >> bottom of it. Making a decision based on this one JIRA seems premature. > >> > >> > In any case, security is meaningless without ZK 3.4, so I am not in > >> > favor of reverting. > >> > >> > >> Likewise. > >> > >> Whomever reverts the main build to ZK to 3.3 while retaining 3.4 for > >> security would have to add shims for the NIO server constructor. There > is > >> also a problematic Enum change. > >> > >> Best regards, > >> > >> - Andy > >> > >> Problems worthy of attack prove their worth by hitting back. - Piet > Hein > >> (via Tom White) > >> > >> > >> ----- Original Message ----- > >> > From: Gary Helmling <[email protected]> > >> > To: [email protected] > >> > Cc: > >> > Sent: Friday, January 27, 2012 9:16 AM > >> > Subject: Re: rethinking zookeeper version > >> > > >> > As I recall, there were other API changes in zk 3.3 -> 3.4 that > > would > >> > make reverting a bit more complicated. Like the change of > >> > NIOServerCnxn.Factory -> NIOServerCnxnFactory (refactor to top > > level > >> > class). So reverting while keeping 3.4 usage for security would > >> > require more work to put in place some kind of shim layer. > >> > > >> > In any case, security is meaningless without ZK 3.4, so I am not in > >> > favor of reverting. I haven't been tracking 3.4 development > > closely, > >> > so I don't know how much pain bugs in that release have been > > causing. > >> > But 3.3 has had issues too. I was just bit by ZOOKEEPER-1208 last > >> > week on a running cluster. Of course this issue is fixed in 3.3.4 > and > >> > 3.4.0. But that would by my opinion for any current issues we're > >> > seeing with 3.4 as well -- let's try to get them fixed and move on > >> > instead of putting effort into backtracking for a temporary solution. > >> > > >> > --gh > >> > > >> > > >> > On Fri, Jan 27, 2012 at 7:50 AM, Ted Yu <[email protected]> > > wrote: > >> >> That's what we have done for internal repository. > >> >> > >> >> Some of the bugs in 3.4.x are hard to reproduce, track down and > > fix. > >> >> > >> >> Of course, Gary and Andrew's opinions are important. > >> >> > >> >> On Fri, Jan 27, 2012 at 7:23 AM, Todd Lipcon > > <[email protected]> > >> > wrote: > >> >> > >> >>> At one point I had proposed making the ZK dependency switch > > only for > >> >>> the security profile in the pom. The ZK 3.4.x series has been > > buggy so > >> >>> far - I'm sure it will stabilize within month or two, > > but I'd > >> > be +1 > >> >>> on reverting the non-secure build to 3.3.x in the meantime. > >> >>> > >> >>> -Todd > >> >>> > >> >>> On Thu, Jan 26, 2012 at 8:25 PM, Ted Yu > > <[email protected]> > >> > wrote: > >> >>> > HBase 0.92 is using zookeeper 3.4.2 > >> >>> > > >> >>> > Maybe some of you have seen this JIRA > >> >>> > https://issues.apache.org/jira/browse/ZOOKEEPER-1367 > >> >>> > It looks like a serious issue. > >> >>> > > >> >>> > Cheers > >> >>> > >> >>> > >> >>> > >> >>> -- > >> >>> Todd Lipcon > >> >>> Software Engineer, Cloudera > >> >>> > >> > > >> > > > -- // Jonathan Hsieh (shay) // Software Engineer, Cloudera // [email protected]
