Initial results from HBASE-5074, support checksums in HBase block cache, look promising.
Shall we discuss whether 0.94 should include it (assuming Dhruba can finish the feature in Feb.) ? Cheers On Fri, Jan 27, 2012 at 6:27 PM, lars hofhansl <lhofha...@yahoo.com> wrote: > Salesforce will jumpstart with 0.94. :) Our dev cluster has the current > trunk on it. > > OK let's just add these to 0.94: > HBASE-4218 > > HBASE-4608 > HBASE-5128 > script necessary to do HBASE-5293 in 0.96 > > > And defer the rest to 0.96, and branch 0.94 soon'ish. > Do you think you can do a 0.92 and trunk version of HBASE-5128? > > -- Lars > > > ________________________________ > From: Jonathan Hsieh <j...@cloudera.com> > To: Matt Corgan <mcor...@hotpads.com> > Cc: lars hofhansl <lhofha...@yahoo.com>; dev <dev@hbase.apache.org> > Sent: Friday, January 27, 2012 6:11 PM > Subject: Re: hbase 0.94.0 > > Lars, Matt, > > I like Matt's suggestion -- as long as it is not a burden let's keep it in. > If it becomes one, let's do what is best for the project. > > If Cloudera needs to do the upgrade path, we'll provide one. If it doesn't > go to the apache repo, we'll most likely open source it on github or > something, or "keep it on the jira" like some of the repair ruby scripts. > > Question -- to the Trend/SalesForce/FB folks can you talk about your plans > for upgrades? Are you guys moving from 0.89/0.90ish to 0.92 already? or > are you guys going to have to do the direct upgrade to 0.94 from 0.90 land > as well? > > Jon. > > > On Fri, Jan 27, 2012 at 12:06 PM, Matt Corgan <mcor...@hotpads.com> wrote: > > > The reason I brought it up is now that Mikhail committed the data block > > encoding I was going to take a stab at adding the prefix trie encoding I > > was working on this past summer. My plan is to first make a minimally > > invasive patch to prove correctness. But, after that there will probably > > be some big performance gains to be had from reworking some of the things > > like KeyValueScanner which I would not have the courage to get working > with > > v1. > > > > So, that was why I asked, but all of that is still more hypothetical than > > real, and I don't even know if the first part will be done before > branching > > .94 at the end of February. Makes sense to me to not delete v1 until > > there's a good reason to, which it doesn't look like we have yet. If I > get > > to the point where v1 is halting progress then we can reevaluate based on > > more specific issues. Maybe none of the prefix trie will even make > .94... > > > > ..sent from my phone > > On Jan 27, 2012 1:55 PM, "lars hofhansl" <lhofha...@yahoo.com> wrote: > > > >> Hey Jon, > >> > >> understood. Makes 0.94 hard, though. If we decide now to have a 0.90 to > >> 0.94 upgrade path and then timing does not work out and nobody signs up > for > >> the testing because it 0.92 is more convenient we'd have gone through > this > >> for nothing. > >> > >> So... Thinking about this more I am -1 on supporting an official upgrade > >> path other then from one release to the next. > >> That said, we do not have to break things intentionally. > >> > >> > >> I'm fine pushing HBASE-2600 and HFile v1 removal out of 0.94... As long > >> as we won't havethe same argument for 0.96 :) > >> > >> And I am not aware of any file compatibility issues. > >> > >> > >> We can also leave the 0.92 migration code in, but not officially support > >> it as 0.90 to 0.94 path. > >> Then CDH4 can make sure (if needed) that it is all working. > >> > >> > >> Does that work for you guys? > >> > >> -- Lars > >> > >> > >> > >> ________________________________ > >> From: Jonathan Hsieh <j...@cloudera.com> > >> To: dev@hbase.apache.org; lars hofhansl <lhofha...@yahoo.com> > >> Sent: Friday, January 27, 2012 10:10 AM > >> Subject: Re: hbase 0.94.0 > >> > >> > >> Lars, > >> > >> The upcoming CDH4 Beta HBase will be based on the latest hotness, > 0.92.0. > >> A CDH4 GA HBase will have to have an upgrade path from CDH3 HBase. If > that > >> HBase is 0.92 based, we'll test that, and if timing works out and we > decide > >> 0.94, we'll have to have a path (0.90->0.94) for than and will test > that. > >> > >> HBASE-2600 is a big change of encoding of meta, while my understanding > is > >> that 0.90->0,92 is a graceful HFile format conversion. Are there are > >> things currently in trunk that further break compatiblity of the file > >> format? (what jiras?) > >> > >> Jon. > >> > >> > >> On Fri, Jan 27, 2012 at 9:16 AM, lars hofhansl <lhofha...@yahoo.com> > >> wrote: > >> > >> Not removing code for upgrade is fine. > >> > > >> >Todd, is Cloudera signing up for testing this path (0.90 to 0.94)? > >> > > >> > > >> >Stack, what's your feeling w.r.t. to HBASE-2600, will keeping the 0.90 > >> -> 0.92 migration path > >> >make the migration code for HBASE-2600 (much) more complicated in 0.94? > >> > > >> > > >> >-- Lars > >> > > >> > > >> > > >> >----- Original Message ----- > >> >From: Stack <st...@duboce.net> > >> >To: dev@hbase.apache.org > >> >Cc: > >> >Sent: Friday, January 27, 2012 9:02 AM > >> >Subject: Re: hbase 0.94.0 > >> > > >> > > >> >On Fri, Jan 27, 2012 at 8:42 AM, Stack <st...@duboce.net> wrote: > >> >> In this particular case, there was no explicit migration step needed > >> >> going 0.90 to 0.92. Upgrading from 0.90 to 0.94 should just be > >> >> running the 0.92 to 0.94 migration script (if there is one). > >> >> > >> > > >> >Thinking more, the above only really holds if we keep the .META. > >> >migration code that runs in 0.92 on startup on into 0.94 (I have a > >> >patch here where I'm removing it... I should put this bit of code > >> >back). > >> > > >> >I see Todd that you vote against removing hfile v1 in 0.94. To make > >> >going from CDH3 to CDH4, we should not purge any migrating code or old > >> >version support. > >> > > >> >I'd be fine w/ that. We'll need to work hard to ensure it taking it > >> >on as a principal for 0.94. Ok w/ you "Iron Hand" Lars? > >> > > >> >St.Ack > >> > > >> > > >> > >> > >> -- > >> // Jonathan Hsieh (shay) > >> // Software Engineer, Cloudera > >> > >> // j...@cloudera.com > >> > > > > > -- > // Jonathan Hsieh (shay) > // Software Engineer, Cloudera > // j...@cloudera.com >
