Cool... Does this script still exist somewhere, can we reuse it?
----- Original Message ----- From: Jean-Daniel Cryans <jdcry...@apache.org> To: dev@hbase.apache.org Cc: Sent: Thursday, January 26, 2012 5:36 PM Subject: Re: hbase 0.94.0 The upgrade from 0.20 to 0.89 was kinda like that; it required having all your regions major compacted including catalog tables. Unfortunately the last part was hard to do since shutting down the cluster could do a flush on META or ROOT so we added a script in the 0.20 branch (so it was never even released) to major compact an offline cluster (it could also check if everything was major compacted). Having to upgrade to a point release sounds easy in comparison. J-D On Thu, Jan 26, 2012 at 5:30 PM, lars hofhansl <lhofha...@yahoo.com> wrote: > Seem weird to force folks to install specific point releases to do an > upgrade. But I am certainly not opposed to have this in 0.92.1. > Do you have time to work on this script? Could probably be a ruby script to > be run by the HBase shell. > > > > ----- Original Message ----- > From: Matt Corgan <mcor...@hotpads.com> > To: dev@hbase.apache.org; lars hofhansl <lhofha...@yahoo.com> > Cc: > Sent: Thursday, January 26, 2012 5:21 PM > Subject: Re: hbase 0.94.0 > > It would be a simple, read-only script that iterates through your hbase > directory on hdfs, looks at each file, and prints out any regions or tables > that contain v1 files. Either you've been running 0.92 for a while are are > not likely to have many v1 files, or you are using 0.92 as the "script" to > migrate your hfiles in which case all your regions will need major > compacting. The user can copy/paste the identified regions (or whole > tables) over into the shell to major compact them. After major compacting > all of them, run the script again to verify there are no v1 files. Then > you are safe to upgrade from 0.92 to 0.94. > > Lars - did you mean it would be too late to add a script like that to > 0.92.1 release? If it were included in the next release, then seems like > it would be safe to drop v1 support from 0.94.0. > > > On Thu, Jan 26, 2012 at 4:53 PM, lars hofhansl <lhofha...@yahoo.com> wrote: > >> I fear it will be close to impossible to have an upgrade path from any >> version of HBase to every future version. >> At some point we have to make the cut, or the code will littered with old >> cruft and upgrade logic, not even to speak of the testing overhead >> to verify that all old versions can be upgraded to the latest one. >> >> >> If we only support upgrade from one version to the next we can make sure >> that it is rock solid and think through all the corner cases. >> >> And then we can stop maintaining old code and focus on fixing bugs and >> adding features. >> >> >> I like Matt's idea being able to check that all HFiles did in fact upgrade >> to V2 (that falls into the "rock solid" part). >> And maybe that means it is too late to remove HFileV1 in 0.94. >> >> >> >> ----- Original Message ----- >> From: Jonathan Hsieh <j...@cloudera.com> >> To: dev@hbase.apache.org >> Cc: >> Sent: Thursday, January 26, 2012 4:22 PM >> Subject: Re: hbase 0.94.0 >> >> +1 to having some sort of migration mechanism especially for the files >> side. I say this out of personal interest -- I'm fairly certain that at >> some point I'm going to have to support these upgrades. >> >> Jon. >> >> On Thu, Jan 26, 2012 at 12:25 PM, Jesse Yates <jesse.k.ya...@gmail.com >> >wrote: >> >> > +1 on removing it too, but maybe we should have some sort of upgrade >> > script to help make the switch? >> > >> > I'm thinking something pluggable on both sides of the upgrade, so people >> > can go from version X->Y, rather than having to upgrade first to an >> > intermediate and then to he version they want (right it would be going >> from >> > 0.20->.92->.96, IMO an excessive PITA). >> > >> > Just my two cents... >> > >> > - Jesse Yates >> > >> > Sent from my iPhone. >> > >> > On Jan 26, 2012, at 12:16 PM, lars hofhansl <lhofha...@yahoo.com> wrote: >> > >> > > Good point. >> > > 0.94 is not branched, yet. And I think generally we do not support >> > skipping releases for upgrades. >> > > +1 on removing HFileV1 cruft for 0.94 >> > > >> > > >> > > -- Lars >> > > >> > > >> > > >> > > ________________________________ >> > > From: Matt Corgan <mcor...@hotpads.com> >> > > To: dev@hbase.apache.org; Andrew Purtell <apurt...@apache.org> >> > > Sent: Thursday, January 26, 2012 11:51 AM >> > > Subject: Re: hbase 0.94.0 >> > > >> > > Are there any thoughts about when it is ok to stop being backwards >> > > compatible? Mainly thinking of HFileV1... 0.92 will convert all >> > HFileV1's >> > > to HFileV2's, so it would probably have been ok to delete the code for >> > > HFileV1 in 0.94 and force people to upgrade through 0.92. That didn't >> > > actually happen, so it's looking like folks will be able to go straight >> > > from 0.90 to 0.94. But, perhaps it should be deleted for 0.96? >> > > >> > > >> > > On Thu, Jan 26, 2012 at 11:41 AM, Andrew Purtell <apurt...@apache.org >> > >wrote: >> > > >> > >> Yeah, so >> > >> >> > >> - Security (basically another coprocessor for inclusion in >> mainline, >> > >> like Constraints) >> > >> >> > >> if not in 0.92.1. >> > >> >> > >> Best regards, >> > >> >> > >> - Andy >> > >> >> > >> Problems worthy of attack prove their worth by hitting back. - Piet >> Hein >> > >> (via Tom White) >> > >> >> > >> >> > >>> ________________________________ >> > >>> From: Andrew Purtell <apurt...@apache.org> >> > >>> To: "dev@hbase.apache.org" <dev@hbase.apache.org> >> > >>> Sent: Thursday, January 26, 2012 11:28 AM >> > >>> Subject: Re: hbase 0.94.0 >> > >>> >> > >>> A limited set of changes so we can get it out on that kind of >> > timeframe. >> > >> :-) >> > >>> >> > >>> - Constraints (is ready to go, is a coprocessor, so is in the large >> > >> just a new package to drop in) >> > >>> >> > >>> - Useful utilities for ops: >> > >>> >> > >>> - LoadTestTool (if Ted didn't end up backporting this into 0.92) >> > >>> >> > >>> - The store file locality thing I have mostly done, will finish >> it >> > >>> >> > >>> - Mikhail and crew's ongoing optimizations (HBASE-4218, etc.), the >> > ones >> > >> he considers fully baked >> > >>> >> > >>> I saw wire compatibility mentioned, for 0.96 but perhaps >> > >> optional/transitional code in 0.94 as well. This is something we would >> > try >> > >> out and beat up upon in staging in earnest. >> > >>> >> > >>> Best regards, >> > >>> >> > >>> - Andy >> > >>> >> > >>> Problems worthy of attack prove their worth by hitting back. - Piet >> > Hein >> > >> (via Tom White) >> > >>> >> > >>> >> > >>> >> > >>>> ________________________________ >> > >>>> From: Stack <st...@duboce.net> >> > >>>> To: HBase Dev List <dev@hbase.apache.org> >> > >>>> Sent: Wednesday, January 25, 2012 8:34 PM >> > >>>> Subject: hbase 0.94.0 >> > >>>> >> > >>>> Lets branch end of february? No new features thereafter. Is this >> too >> > >>>> close in? Would be grand if 0.94.0 shipped before hbasecon. What >> > >>>> should 0.94.0 have in it? I don't mind if the list is short. >> > >>>> >> > >>>> Unless someone else wants too, I don't mind being release manager >> > >>>> (will try to run a tighter ship this time around). >> > >>>> >> > >>>> St.Ack >> > >>>> >> > >>>> >> > >>>> >> > >>> >> > >>> >> > >> >> >> >> -- >> // Jonathan Hsieh (shay) >> // Software Engineer, Cloudera >> // j...@cloudera.com >> >> >
