Before long term wire and format compatibility is reached, I am -0 on time based release.
As Nicolas pointed out, FB wants to migrate from 0.89-fb to 0.94 If we do time-based releases without wire and format compatibility, it would consume a lot of our energy satisfying such requirements from different companies. I propose wire and format compatibility as the main theme for 0.96 Cheers On Mon, Jan 30, 2012 at 11:40 AM, Andrew Purtell <[email protected]>wrote: > > Maybe the discussion should be whether 0.94 should be a time-gated or > > feature-gated release? IMO we already have enough good stuff in there > > on the perf front. It will be great if the checksum improvement makes > > it, but if it doesn't, I'd rather have a release than delay for it. > > > Time gated releases makes sense going forward for a few reasons, mark me > down for that option. > > > Best regards, > > - Andy > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > (via Tom White) > > > ----- Original Message ----- > > From: Todd Lipcon <[email protected]> > > To: [email protected] > > Cc: lars hofhansl <[email protected]>; Matt Corgan < > [email protected]> > > Sent: Monday, January 30, 2012 10:44 AM > > Subject: Re: hbase 0.94.0 > > > > On Sun, Jan 29, 2012 at 7:30 AM, Ted Yu <[email protected]> wrote: > >> Initial results from HBASE-5074, support checksums in HBase block > cache, > >> look promising. > >> > >> Shall we discuss whether 0.94 should include it (assuming Dhruba can > finish > >> the feature in Feb.) ? > > > > If it's a time-based release, then there's nothing to discuss. Either > > it's done in time, in which case it's part of 94, or it's not, in > > which case it's part of 96, right? > > > > Maybe the discussion should be whether 0.94 should be a time-gated or > > feature-gated release? IMO we already have enough good stuff in there > > on the perf front. It will be great if the checksum improvement makes > > it, but if it doesn't, I'd rather have a release than delay for it. > > > > -Todd > > > >> > >> Cheers > >> > >> On Fri, Jan 27, 2012 at 6:27 PM, lars hofhansl <[email protected]> > > wrote: > >> > >>> Salesforce will jumpstart with 0.94. :) Our dev cluster has the > current > >>> trunk on it. > >>> > >>> OK let's just add these to 0.94: > >>> HBASE-4218 > >>> > >>> HBASE-4608 > >>> HBASE-5128 > >>> script necessary to do HBASE-5293 in 0.96 > >>> > >>> > >>> And defer the rest to 0.96, and branch 0.94 soon'ish. > >>> Do you think you can do a 0.92 and trunk version of HBASE-5128? > >>> > >>> -- Lars > >>> > >>> > >>> ________________________________ > >>> From: Jonathan Hsieh <[email protected]> > >>> To: Matt Corgan <[email protected]> > >>> Cc: lars hofhansl <[email protected]>; dev > > <[email protected]> > >>> Sent: Friday, January 27, 2012 6:11 PM > >>> Subject: Re: hbase 0.94.0 > >>> > >>> Lars, Matt, > >>> > >>> I like Matt's suggestion -- as long as it is not a burden let's > > keep it in. > >>> If it becomes one, let's do what is best for the project. > >>> > >>> If Cloudera needs to do the upgrade path, we'll provide one. If it > > doesn't > >>> go to the apache repo, we'll most likely open source it on github > > or > >>> something, or "keep it on the jira" like some of the repair > > ruby scripts. > >>> > >>> Question -- to the Trend/SalesForce/FB folks can you talk about your > > plans > >>> for upgrades? Are you guys moving from 0.89/0.90ish to 0.92 already? > > or > >>> are you guys going to have to do the direct upgrade to 0.94 from 0.90 > > land > >>> as well? > >>> > >>> Jon. > >>> > >>> > >>> On Fri, Jan 27, 2012 at 12:06 PM, Matt Corgan > > <[email protected]> wrote: > >>> > >>> > The reason I brought it up is now that Mikhail committed the data > > block > >>> > encoding I was going to take a stab at adding the prefix trie > > encoding I > >>> > was working on this past summer. My plan is to first make a > > minimally > >>> > invasive patch to prove correctness. But, after that there will > > probably > >>> > be some big performance gains to be had from reworking some of the > > things > >>> > like KeyValueScanner which I would not have the courage to get > > working > >>> with > >>> > v1. > >>> > > >>> > So, that was why I asked, but all of that is still more > > hypothetical than > >>> > real, and I don't even know if the first part will be done > > before > >>> branching > >>> > .94 at the end of February. Makes sense to me to not delete v1 > > until > >>> > there's a good reason to, which it doesn't look like we > > have yet. If I > >>> get > >>> > to the point where v1 is halting progress then we can reevaluate > > based on > >>> > more specific issues. Maybe none of the prefix trie will even > > make > >>> .94... > >>> > > >>> > ..sent from my phone > >>> > On Jan 27, 2012 1:55 PM, "lars hofhansl" > > <[email protected]> wrote: > >>> > > >>> >> Hey Jon, > >>> >> > >>> >> understood. Makes 0.94 hard, though. If we decide now to have > > a 0.90 to > >>> >> 0.94 upgrade path and then timing does not work out and nobody > > signs up > >>> for > >>> >> the testing because it 0.92 is more convenient we'd have > > gone through > >>> this > >>> >> for nothing. > >>> >> > >>> >> So... Thinking about this more I am -1 on supporting an > > official upgrade > >>> >> path other then from one release to the next. > >>> >> That said, we do not have to break things intentionally. > >>> >> > >>> >> > >>> >> I'm fine pushing HBASE-2600 and HFile v1 removal out of > > 0.94... As long > >>> >> as we won't havethe same argument for 0.96 :) > >>> >> > >>> >> And I am not aware of any file compatibility issues. > >>> >> > >>> >> > >>> >> We can also leave the 0.92 migration code in, but not > > officially support > >>> >> it as 0.90 to 0.94 path. > >>> >> Then CDH4 can make sure (if needed) that it is all working. > >>> >> > >>> >> > >>> >> Does that work for you guys? > >>> >> > >>> >> -- Lars > >>> >> > >>> >> > >>> >> > >>> >> ________________________________ > >>> >> From: Jonathan Hsieh <[email protected]> > >>> >> To: [email protected]; lars hofhansl > > <[email protected]> > >>> >> Sent: Friday, January 27, 2012 10:10 AM > >>> >> Subject: Re: hbase 0.94.0 > >>> >> > >>> >> > >>> >> Lars, > >>> >> > >>> >> The upcoming CDH4 Beta HBase will be based on the latest > > hotness, > >>> 0.92.0. > >>> >> A CDH4 GA HBase will have to have an upgrade path from CDH3 > > HBase. If > >>> that > >>> >> HBase is 0.92 based, we'll test that, and if timing works > > out and we > >>> decide > >>> >> 0.94, we'll have to have a path (0.90->0.94) for than > > and will test > >>> that. > >>> >> > >>> >> HBASE-2600 is a big change of encoding of meta, while my > > understanding > >>> is > >>> >> that 0.90->0,92 is a graceful HFile format conversion. Are > > there are > >>> >> things currently in trunk that further break compatiblity of > > the file > >>> >> format? (what jiras?) > >>> >> > >>> >> Jon. > >>> >> > >>> >> > >>> >> On Fri, Jan 27, 2012 at 9:16 AM, lars hofhansl > > <[email protected]> > >>> >> wrote: > >>> >> > >>> >> Not removing code for upgrade is fine. > >>> >> > > >>> >> >Todd, is Cloudera signing up for testing this path (0.90 > > to 0.94)? > >>> >> > > >>> >> > > >>> >> >Stack, what's your feeling w.r.t. to HBASE-2600, will > > keeping the 0.90 > >>> >> -> 0.92 migration path > >>> >> >make the migration code for HBASE-2600 (much) more > > complicated in 0.94? > >>> >> > > >>> >> > > >>> >> >-- Lars > >>> >> > > >>> >> > > >>> >> > > >>> >> >----- Original Message ----- > >>> >> >From: Stack <[email protected]> > >>> >> >To: [email protected] > >>> >> >Cc: > >>> >> >Sent: Friday, January 27, 2012 9:02 AM > >>> >> >Subject: Re: hbase 0.94.0 > >>> >> > > >>> >> > > >>> >> >On Fri, Jan 27, 2012 at 8:42 AM, Stack > > <[email protected]> wrote: > >>> >> >> In this particular case, there was no explicit > > migration step needed > >>> >> >> going 0.90 to 0.92. Upgrading from 0.90 to 0.94 > > should just be > >>> >> >> running the 0.92 to 0.94 migration script (if there > > is one). > >>> >> >> > >>> >> > > >>> >> >Thinking more, the above only really holds if we keep the > > .META. > >>> >> >migration code that runs in 0.92 on startup on into 0.94 > > (I have a > >>> >> >patch here where I'm removing it... I should put this > > bit of code > >>> >> >back). > >>> >> > > >>> >> >I see Todd that you vote against removing hfile v1 in > > 0.94. To make > >>> >> >going from CDH3 to CDH4, we should not purge any migrating > > code or old > >>> >> >version support. > >>> >> > > >>> >> >I'd be fine w/ that. We'll need to work hard to > > ensure it taking it > >>> >> >on as a principal for 0.94. Ok w/ you "Iron > > Hand" Lars? > >>> >> > > >>> >> >St.Ack > >>> >> > > >>> >> > > >>> >> > >>> >> > >>> >> -- > >>> >> // Jonathan Hsieh (shay) > >>> >> // Software Engineer, Cloudera > >>> >> > >>> >> // [email protected] > >>> >> > >>> > > >>> > >>> > >>> -- > >>> // Jonathan Hsieh (shay) > >>> // Software Engineer, Cloudera > >>> // [email protected] > >>> > > > > > > > > -- > > Todd Lipcon > > Software Engineer, Cloudera > > >
