On Wed, Mar 15, 2017 at 5:51 PM, Ted Yu <yuzhih...@gmail.com> wrote: > Thanks for the vote, Josh. > > So far there have been 3 +1's (Enis', mine and Josh's) > > Should we start another thread on the procedure of merging or use this > thread ? > Background: > HBASE-7912 branch is way out of date. > Latest rounds of mega patch were based on then-current master branch. > > 1. Vlad is running this VOTE, not you. 2. Don't you need 3 PMC votes to merge? I count 2. 3. You almost derailed the merge with your earlier interjection rushing to take us in the wrong direction; you'd think you'd learn from your past experience but here you are again w/ haste and rash summary. 4. This vote has been awkward. Josh's note is considered and careful. You'd think it could hang out there a while to see if draws a response rather than rush to close.
I thought I was reviewing a patch that was against master, not some version of master from eons ago. Thats a problem. When was the last rebase? St.Ack > Thanks > > On Wed, Mar 15, 2017 at 4:22 PM, Josh Elser <els...@apache.org> wrote: > > > Spent the day playing around with this on 6 nodes. > > > > I've found some rough edges: some known (the docs blocker Vlad pointed > > out) and some that I think are unknown (tooling is rough -- partially > from > > wrong help messages and, I think, changes in design like Master-submitted > > MR jobs). > > > > But, Stack's assessment (and Andrew's reminder) that further tweaking > > would just throw us back into another review cycle is a real concern. > It's > > unfortunate that this feature has lingered so long aside of master to get > > to this point, but I don't see any realistic resolution to this problem > > than a merge. In the future, this is something we'll have to try harder > to > > avoid letting happen (looking in the mirror with quota work...) > > > > Thankfully, I can help out with development/review on the outstanding > > blockers (notably, the two I pruned from Vlad's original of five -- the > > other three still seem to be improvements). In addition to these > blockers, > > I believe the documentation *must* be updated before a release to note > that > > this feature is still growing -- it does not feel like a quality feature > > that I've come to expect from this community. This isn't a knock on Vlad > > and company; this is a hard problem and one that I could not have done > > better given time constraints, but it is also one that users will demand > > simplicity and the utmost correctness around. To this end, I will also > try > > to help out to smooth out these issues in the following 2.x release. > > > > So, this leaves me to say: +1 to merge with the caveat that the docs are > > updated to make sure that any known, user-pitfalls are clearly > documented. > > This vote also comes with as real of a promise that I can make to help > > avoid any issues with this feature that would prevent a 2.0 release. > > > > Thanks to all the giants whose shoulders I'm standing on to be able to > > make this vote. > > > > Andrew Purtell wrote: > > > >> Great, and I changed my vote to -0 because Stack made a good argument > that > >> making more changes would invalidate review up to this point, and I > trust > >> this will be resolved before release. > >> > >> On Tue, Mar 14, 2017 at 4:29 PM, Josh Elser<els...@apache.org> wrote: > >> > >> Sorry Andrew, let me clarify as that didn't come out right. > >>> > >>> I didn't mean that isn't a conversation worth having _now_, just that I > >>> was intentionally avoiding it in my previous email because I didn't > >>> understand the scope of those issues that Vlad had identified. I wanted > >>> to > >>> better understand what they were really meant for a user before coming > to > >>> my own decision about whether or not I think they are blockers. > >>> > >>> That aside, I would agree with you that HBASE-15227 sounds like a real > >>> blocker. Forcing a new full backup for every table sounds really bad -- > >>> that's not the kind of experience we'd want a user to have. > >>> > >>> Andrew Purtell wrote: > >>> > >>> I'm going to intentional avoid addressing the discussion of shipping > >>>> partial features (related, but not relevant at the moment). > >>>> > >>>> Then we are not having the same conversation, because it is precisely > >>>> because this is a vote for this feature to go into 2.0, which is > already > >>>> overdue, so should be released yesterday, that I took mention of > >>>> "blocker" > >>>> at face value. At least one of them seems to certainly approach this > >>>> definition. It will be not user friendly, to say the least, to use > this > >>>> in > >>>> a large scale deployment with HBASE-15227 unfinished. HBASE-15227 > >>>> currently > >>>> has a severity of BLOCKER. Despite what is going on in our politics, > >>>> words > >>>> matter and we do not get to redefine them for convenience. > >>>> > >>>> Once this work is merged in, how is HBASE-15227 not a blocker for 2.0? > >>>> Because Vlad offered to reduce its severity to make me feel better? > >>>> Currently the description on the issue is "System must be tolerant to > >>>> faults. Backup operations MUST be atomic (no partial completion state > in > >>>> system table)" Sounds like a blocker to me, indeed. It is an honest > >>>> assessment and I don't think anyone is doing the community a favor by > >>>> trying to walk that back. > >>>> > >>>> > >>>> On Tue, Mar 14, 2017 at 1:57 PM, Josh Elser<els...@apache.org> > wrote: > >>>> > >>>> I took a moment to read through the "blockers" as originally > identified > >>>> by > >>>> > >>>>> Vlad, and (to echo Enis' take) I read the majority of them as being > >>>>> blockers not for the next release, but for a "full-fledged feature". > >>>>> I'm > >>>>> going to intentional avoid addressing the discussion of shipping > >>>>> partial > >>>>> features (related, but not relevant at the moment). > >>>>> > >>>>> HBASE-15227 is actually the one that bothers me the most, with > >>>>> HBASE-17133 > >>>>> coming in close behind. Vlad, is there any documentation you can > point > >>>>> me > >>>>> to about what the current issues are with the current implementation? > >>>>> For > >>>>> example, what happens now if the system has some kind of "partial > >>>>> completion state"? > >>>>> > >>>>> Documentation is one of those that is really hard to judge. We want > to > >>>>> get > >>>>> this code out for people to use (and to free up our strained dev > >>>>> resources), but what good is some feature if the docs are > >>>>> missing/incomplete? > >>>>> > >>>>> I think I could stomach the docs being inaccurate (with a clear > >>>>> disclaimer > >>>>> that the chapter is incomplete -- that's a 5min task). But, I think I > >>>>> need > >>>>> an answer about how the feature handles our common dist-sys category > of > >>>>> problems before I can consider whether I'm ok with the feature > hitting > >>>>> 2.0... > >>>>> > >>>>> I'll also try to throw up a few nodes and play with it to address the > >>>>> problem as an (ignorant) user ;) > >>>>> > >>>>> > >>>>> Andrew Purtell wrote: > >>>>> > >>>>> I don't like that issues were identified as "blockers" but now there > is > >>>>> > >>>>>> an > >>>>>> attempt to walk that back. > >>>>>> > >>>>>> I don't like that development of this feature has lingered for a > long > >>>>>> time > >>>>>> in this unfinished state when this work could have been done by now, > >>>>>> now > >>>>>> that we are trying to get a 2.0 out the door. Because this is a > >>>>>> volunteer > >>>>>> project I cannot make any demand that it should be done, but I can > >>>>>> certainly look at the current state and be nonplussed. This will be > >>>>>> yet > >>>>>> another half finished thing in 2.0 when this merge happens. Promises > >>>>>> to > >>>>>> finish the unfinished work are nice but not currency. Commits are > >>>>>> currency. > >>>>>> I hope at least the fault tolerance changes can be completed and > >>>>>> committed > >>>>>> before we spin a 2.0 RC, and without causing a 2.0 release to slip > >>>>>> further. > >>>>>> > >>>>>> Also, marking something experimental should be done on the merits of > >>>>>> that > >>>>>> evaluation, not simply to justify dropping unfinished work into a > >>>>>> release > >>>>>> branch. > >>>>>> > >>>>>> I will change my vote to -0. > >>>>>> > >>>>>> > >>>>>> On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar<e...@apache.org> > >>>>>> wrote: > >>>>>> > >>>>>> I think there is some misconception of using the term "blockers" for > >>>>>> > >>>>>> referring to those jiras. My understanding is that those three jiras > >>>>>>> are > >>>>>>> blockers for the backup functionality to be more mature and more > >>>>>>> usable. > >>>>>>> But they are not release blockers. Let's say we merged the code in, > >>>>>>> and > >>>>>>> for > >>>>>>> some reason those did not get addressed in time. We can still do > the > >>>>>>> 2.0 > >>>>>>> release without having to wait for the commits. We can instead mark > >>>>>>> the > >>>>>>> "backup" feature as experimental with known issues and go on with > the > >>>>>>> release. In that sense they are not real release blockers. > >>>>>>> > >>>>>>> We are proposing the merge at this time because of the above that > >>>>>>> maintaining this in a branch is becoming extremely costly and not > >>>>>>> productive for the HBase community. Realistically, we cannot have > the > >>>>>>> luxury of having to wait another couple of months and doing yet > >>>>>>> another > >>>>>>> giant round of reviews because the code base is a moving target. > >>>>>>> > >>>>>>> Enis > >>>>>>> > >>>>>>> On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das<d...@hortonworks.com> > >>>>>>> wrote: > >>>>>>> > >>>>>>> Vlad, on the first point, I think what Stack is saying is that > >>>>>>> creating > >>>>>>> > >>>>>>> the new branch (as Ted did) ignores the feedback incorporated thus > >>>>>>>> far > >>>>>>>> in > >>>>>>>> the iterations of the mega-patch. That's a wrong way to go. > >>>>>>>> On the separation into a backup module, again, that was reverted > to > >>>>>>>> ease > >>>>>>>> reviews of the mega-patch, and was noted as work to be done > later. I > >>>>>>>> > >>>>>>>> think > >>>>>>>> > >>>>>>> Stack just wants to make the list of remaining work more complete > by > >>>>>>> > >>>>>>>> citing > >>>>>>>> > >>>>>>> that as pending work. > >>>>>>> > >>>>>>>> ________________________________________ > >>>>>>>> From: Vladimir Rodionov<vladrodio...@gmail.com> > >>>>>>>> Sent: Monday, March 13, 2017 3:09 PM > >>>>>>>> To: dev@hbase.apache.org > >>>>>>>> Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote > >>>>>>>> closing > >>>>>>>> 3/11/2017 > >>>>>>>> > >>>>>>>> It ignores the feedback > >>>>>>>> > >>>>>>>> If I "ignore" feedback, I put my comment - why? I am always open > >>>>>>>>> for > >>>>>>>>> > >>>>>>>>> further discussions. If reviewer does not insist on a particular > >>>>>>>> request > >>>>>>>> > >>>>>>>> - > >>>>>>>> > >>>>>>> it will be dropped. I think it is fair. > >>>>>>> > >>>>>>>> he list is incomplete because a bunch of > >>>>>>>> > >>>>>>>> follow-ons came of the review cycle (including moving > backup/restore > >>>>>>>>> > >>>>>>>>>> out > >>>>>>>>>> > >>>>>>>>> of > >>>>>>>> > >>>>>>>> core to live in its own module). > >>>>>>>> > >>>>>>>>> For those who were not following our lengthy conversation on a > >>>>>>>>> review > >>>>>>>>> > >>>>>>>>> board, separation of a backup code into a separate module has > been > >>>>>>>> done last year, but has been reverted back by request of a > reviewer. > >>>>>>>> > >>>>>>>> > >>>>>>>> -Vladimir > >>>>>>>> > >>>>>>>> On Mon, Mar 13, 2017 at 2:23 PM, Stack<st...@duboce.net> > wrote: > >>>>>>>> > >>>>>>>> On Fri, Mar 10, 2017 at 9:09 PM, Stack<st...@duboce.net> > wrote: > >>>>>>>> > >>>>>>>> On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu<yuzhih...@gmail.com> > >>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>> HBASE-14123 branch has been created, with Vlad's mega patch v61. > >>>>>>>>>> > >>>>>>>>>> The patch put up for VOTE here was done on a branch. The call to > >>>>>>>>>>> > >>>>>>>>>>> merge > >>>>>>>>>> > >>>>>>>>> seems to have been premature given the many cycles of review and > >>>>>>>> test > >>>>>>>> > >>>>>>>> that > >>>>>>>>> > >>>>>>>>> happened subsequent (The cycles burned a bunch of dev resource). > >>>>>>>>> > >>>>>>>>>> The patch as is is now in a state where it is too big for our > >>>>>>>>>> infra; > >>>>>>>>>> > >>>>>>>>>> rb > >>>>>>>>>> > >>>>>>>>> and JIRA are creaking under the size and # of iterations. > >>>>>>>> > >>>>>>>> Adding finish of new JIRAs to this merge implies a new round of > >>>>>>>>> > >>>>>>>>>> review > >>>>>>>>>> > >>>>>>>>> and > >>>>>>>> > >>>>>>>> test of an already massive patch. Who is going to do this work? > >>>>>>>>> > >>>>>>>>>> Going back to a new branch seems wrong route to take. > >>>>>>>>>> > >>>>>>>>>> St.Ack > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> To be more explicit, this patch was developed on a branch and > >>>>>>>>>> then a > >>>>>>>>>> > >>>>>>>>>> bunch > >>>>>>>>> > >>>>>>>> of dev resources were burned getting it into a state where it > could > >>>>>>>> be > >>>>>>>> > >>>>>>>>> merged to master. Going back to a branch to bulk up the merge so > it > >>>>>>>>> includes more JIRAs than the many it already incorporates is the > >>>>>>>>> wrong > >>>>>>>>> direction for us to be headed in. It ignores the feedback given > and > >>>>>>>>> the > >>>>>>>>> work done by Vladimir slimming down an already over-broad scope. > It > >>>>>>>>> is > >>>>>>>>> > >>>>>>>>> also > >>>>>>>>> > >>>>>>>> predicated on abundant review and testing resource being on tap to > >>>>>>>> > >>>>>>>>> cycle > >>>>>>>>> > >>>>>>>> on > >>>>>>>> > >>>>>>>> a feature that is useful, but non-core. > >>>>>>>> > >>>>>>>>> The patch is ready for merge IMO. Geoffrey makes a nice list of > >>>>>>>>> what > >>>>>>>>> is > >>>>>>>>> still to do though IIRC, the list is incomplete because a bunch > of > >>>>>>>>> follow-ons came of the review cycle (including moving > >>>>>>>>> backup/restore > >>>>>>>>> > >>>>>>>>> out > >>>>>>>>> > >>>>>>>> of > >>>>>>>> > >>>>>>>> core to live in its own module). > >>>>>>>> > >>>>>>>>> The patch needs three votes to merge. I am not against merge but > I > >>>>>>>>> am > >>>>>>>>> > >>>>>>>>> not > >>>>>>>>> > >>>>>>>> voting for the patch because I do have any more time to spend on > >>>>>>>> this > >>>>>>>> > >>>>>>>> non-core feature and feel that a vote will have me assume a > >>>>>>>>> > >>>>>>>>> responsibility > >>>>>>>>> > >>>>>>>> I will not shirk. > >>>>>>>> > >>>>>>>>> S > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> FYI > >>>>>>>>> > >>>>>>>>>> On Fri, Mar 10, 2017 at 3:30 PM, Ted Yu<yuzhih...@gmail.com> > >>>>>>>>>>> > >>>>>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>> Thanks for the feedback, Andrew. > >>>>>>>>> How about the following plan: > >>>>>>>>> > >>>>>>>>>> create branch HBASE-14123 off of master with mega patch v61 as > the > >>>>>>>>>>>> > >>>>>>>>>>>> first > >>>>>>>>>>>> > >>>>>>>>>>> commit (reviewed by Stack and Enis) > >>>>>>>>>> > >>>>>>>>>> Vlad and I continue development (the 3 blockers) based on > >>>>>>>>>>> > >>>>>>>>>>>> HBASE-14123 > >>>>>>>>>>>> > >>>>>>>>>>> branch > >>>>>>>>>> when all of the blockers get +1 and merged into HBASE-14123 > >>>>>>>>>> > >>>>>>>>>>> branch, > >>>>>>>>>>>> > >>>>>>>>>>> we > >>>>>>>>>> > >>>>>>>>> propose to community for merging into branch-2 (master branch, if > >>>>>>>> > >>>>>>>>> branch-2 > >>>>>>>>>> > >>>>>>>>>>> doesn't materialize for whatever reason) again > >>>>>>>>>>> > >>>>>>>>>>>> Cheers > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> On Fri, Mar 10, 2017 at 3:01 PM, Andrew Purtell< > >>>>>>>>>>>> > >>>>>>>>>>>> apurt...@apache.org> > >>>>>>>>>>>> > >>>>>>>>>>> wrote: > >>>>>>>>>> Thanks for the offer but I like that you were honest about > >>>>>>>>>> > >>>>>>>>>>> compiling > >>>>>>>>>>>> > >>>>>>>>>>>> a > >>>>>>>>>>> > >>>>>>>>>> list > >>>>>>>>> > >>>>>>>>>> of issues that you thought were blockers for release. Since this > >>>>>>>>>>> > >>>>>>>>>>>> proposal > >>>>>>>>>>>>> > >>>>>>>>>>>> is a merge into 2.0, and we are trying to release 2.0, I am -1 > >>>>>>>>>>>> on > >>>>>>>>>>>> this > >>>>>>>>>>>> > >>>>>>>>>>>> merge until those blockers are addressed. > >>>>>>>>>>> I had a look at the list. > >>>>>>>>>>> > >>>>>>>>>>>> I think the documentation issue is important but not actually > a > >>>>>>>>>>>>> > >>>>>>>>>>>>> blocker. > >>>>>>>>>>>>> > >>>>>>>>>>>> That may be a controversial opinion, but documentation can be > >>>>>>>>>>>> back-filled > >>>>>>>>>>>> worst case. So take HBASE-17133 off the list. > >>>>>>>>>>>> > >>>>>>>>>>>> Remaining are effectively HBASE-14417, HBASE-14141, and > >>>>>>>>>>>>> > >>>>>>>>>>>>> HBASE-15227. > >>>>>>>>>>>>> > >>>>>>>>>>>> They > >>>>>>>>>>> > >>>>>>>>>> all have patches attached to the respective JIRAs so completing > >>>>>>>>>> > >>>>>>>>>>> this > >>>>>>>>>>>> > >>>>>>>>>>>> work > >>>>>>>>>>> > >>>>>>>>>> won't be onerous. Get these committed and I will lift my -1. The > >>>>>>>>>> > >>>>>>>>>>> others > >>>>>>>>>>>> > >>>>>>>>>>>> who > >>>>>>>>>>> voted +1 on this thread surely can help with that. > >>>>>>>>>>> > >>>>>>>>>>>> Thanks. > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Fri, Mar 10, 2017 at 2:32 PM, Vladimir Rodionov< > >>>>>>>>>>>>> vladrodio...@gmail.com> > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>> No problem I will downgrade Blockers to Majors if it scares > >>>>>>>>>>>>> you, > >>>>>>>>>>>>> > >>>>>>>>>>>>> Andrew > >>>>>>>>>>>> > >>>>>>>>>>> [image: [image: 🙂]] > >>>>>>>>> > >>>>>>>>> Sent from my iPhone > >>>>>>>>>>>> > >>>>>>>>>>>>> On Mar 10, 2017, at 1:52 PM, Andrew Purtell< > >>>>>>>>>>>>>> apurt...@apache.org > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>> I know the merge of this feature has lagged substantially. I > >>>>>>>>>> > >>>>>>>>>>> think > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> that > >>>>>>>>>>>>> > >>>>>>>>>>>> is > >>>>>>>>>>> > >>>>>>>>>>>> regrettable but on another thread we are lamenting that 2.0 > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> is > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> already > >>>>>>>>>>>>> > >>>>>>>>>>>> late. Unless I misunderstand, this is a proposal to merge > >>>>>>>>> > >>>>>>>>>> something > >>>>>>>>>>>>> with > >>>>>>>>>>>>> > >>>>>>>>>>>> known blockers into trunk before we branch it for 2.0 which > >>>>>>>>>>> > >>>>>>>>>>>> will > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> effectively prevent that release because these blockers will > >>>>>>>>>>>>> > >>>>>>>>>>>> be > >>>>>>>>>> > >>>>>>>>>>> there. I > >>>>>>>>>>>>> > >>>>>>>>>>>> am > >>>>>>>>> > >>>>>>>>>> inclined to veto. Probably we should not propose branch > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> merges > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> into > >>>>>>>>>>>>> > >>>>>>>>>>>> code > >>>>>>>>> > >>>>>>>>>> we > >>>>>>>>>>> > >>>>>>>>>>>> are trying to get out the door with known blockers. Why not > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> do > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> that > >>>>>>>>>>>>> > >>>>>>>>>>>> work > >>>>>>>>> > >>>>>>>>>> first? It seems an obvious question. Perhaps I am missing > >>>>>>>>>>> > >>>>>>>>>>>> something. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> If we can branch for 2.0 now and then merge this, and not > >>>>>>>>>>>>> into > >>>>>>>>>>>>> the > >>>>>>>>>>>>> > >>>>>>>>>>>> 2.0 > >>>>>>>>> > >>>>>>>>>> branch, I would vote +1 for branch merge even with known > >>>>>>>>>>> > >>>>>>>>>>>> blockers > >>>>>>>>>>>>> pending. > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>>> On Fri, Mar 10, 2017 at 1:42 PM, Vladimir Rodionov< > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> vladrodio...@gmail.com> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> They are not blockers for merge - only for 2.0. GA > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> As I said already the feature is usable right now > >>>>>>>>>>>>>>>> We would like to continue working on master and we would > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> like > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> to > >>>>>>>>>>>>>> > >>>>>>>>>>>>> see > >>>>>>>> > >>>>>>>>> a > >>>>>>>>>> > >>>>>>>>>>> commitment from community > >>>>>>>>>>>>> > >>>>>>>>>>>>>> Sent from my iPhone > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> On Mar 10, 2017, at 11:16 AM, Andrew Purtell< > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> apurt...@apache.org > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>> Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 > >>>>>>>>>>> > >>>>>>>>>>>> release. > >>>>>>>>>>>>>>>> If we have identified blockers, why merge this before they > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> are > >>>>>>>>>>>>> > >>>>>>>>>>>>>> in? > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> Otherwise we can't release 2.0, and it is overdue. > >>>>>>>>>> > >>>>>>>>>>> On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov< > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> vladrodio...@gmail.com> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Hello, HBase folks > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> For your consideration today is Backup/Restore feature > for > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Apache > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> HBAse > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> 2.0. > >>>>>>>>>>>>> > >>>>>>>>>>>>>> Backup code is available as a mega patch in HBASE-14123 > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> (v61), > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> applies > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> cleanly to the current master, all test PASS, patch has no > >>>>>>>>>> > >>>>>>>>>>> other > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> issues. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> The patch has gone through numerous rounds of code reviews > >>>>>>>>>>> > >>>>>>>>>>>> and > >>>>>>>>>>>>>>>> has > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> probably > >>>>>>>>>> > >>>>>>>>>>> the most lengthy discussion thread on Apache JIRA > >>>>>>>>>>>>> > >>>>>>>>>>>>>> (HBASE-14123) > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> :) > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> The work has been split into 3 phases (HBASE-14030, 14123, > >>>>>>>>>>> > >>>>>>>>>>>> 14414) > >>>>>>>>>>>>> > >>>>>>>>>>>>>> Two > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> first > >>>>>>>>>>>>> > >>>>>>>>>>>>>> are complete, third one is still in progress. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> *** Summary of work HBASE-14123 > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> The new feature introduces new command-line extensions > to > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> hbase > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> command > >>>>>>>>>> > >>>>>>>>>>> and, from the client side, is accessible through > >>>>>>>>>>>>> > >>>>>>>>>>>>>> command-line > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> only > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Operations: > >>>>>>>>>> > >>>>>>>>>>> * Create full backup on a list of tables or backup set > >>>>>>>>>>>>> > >>>>>>>>>>>>>> * Create incremental backup image for table list or backup > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> set > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> * Restore list of tables from a given backup image > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> * Show current backup progress > >>>>>>>>>> > >>>>>>>>>>> * Delete backup image and all related images > >>>>>>>>>>>>>>>>>> * Show history of backups > >>>>>>>>>>>>>>>>>> * Backup set operations: create backup set, add/remove > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> table > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> to/from > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> backup > >>>>>>>>> > >>>>>>>>>> set, etc > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> In the current implementation, the feature is already > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> usable, > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> meaning > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> that > >>>>>>>>>> > >>>>>>>>>>> users can backup tables and restore them using provided > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> command-line > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> tools. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Both: full and incremental backups are supported. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> This work is based on original work of IBM team > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> (HBASE-7912). > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> The > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> full > >>>>>>>>>> > >>>>>>>>>>> list > >>>>>>>>>>>>> > >>>>>>>>>>>>>> of JIRAs included in this mega patch can be found in three > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> umbrella > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> JIRAs: > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and > >>>>>>>>>>>>> > >>>>>>>>>>>>>> HBASE-14414 > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> (Phase 3 > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> - > >>>>>>>>> > >>>>>>>>>> all > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> resolved ones made it into the patch) > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> *** What are the remaining work items > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> All remaining items can be found in Phase 3 umbrella > JIRA: > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> HBASE-14414. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> They are split into 3 groups: BLOCKER, CRITICAL, MAJOR > >>>>>>>>>>>>>>>> Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> release. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> ***** BLOCKER > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> * HBASE-14417 Incremental backup and bulk loading ( Patch > >>>>>>>>>>>>> > >>>>>>>>>>>>>> available) > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> images > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> backup > >>>>>>>>>>> > >>>>>>>>>>>> to > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> include only edits from backup tables (Patch available) > >>>>>>>>>>>>> > >>>>>>>>>>>>>> * HBASE-17133 Backup documentation > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> * HBASE-15227 Fault tolerance support > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> ***** CRITICAL > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> * HBASE-16465 Disable split/merges during backup > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> We have umbrella JIRA (HBASE-14414) to track all the > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> remaining > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> work > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> All the BLOCKER and CRITICAL JIRAs currently in open state > >>>>>>>>>> > >>>>>>>>>>> will > >>>>>>>>>>>>> > >>>>>>>>>>>>>> be > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> implemented by 2.0 release time. Some MAJOR too, but it > >>>>>>>>>>> > >>>>>>>>>>>> depends > >>>>>>>>>>>>> > >>>>>>>>>>>>>> on > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> resource > >>>>>>>>>>> > >>>>>>>>>>>> availability > >>>>>>>>>>>>> > >>>>>>>>>>>>>> The former development branch (HBASE-7912) is obsolete and > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> will > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> be > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> closed/deleted after the merge. > >>>>>>>>>>> > >>>>>>>>>>>> We want backup to be a GA feature in 2.0 > >>>>>>>>>>>>> > >>>>>>>>>>>>>> We are going to support full backward compatibility for > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> backup > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> tool in > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> 2.0 > >>>>>>>>>> > >>>>>>>>>>> and onwards. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> **** Configuration > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Backup is disabled, by default. To enable it, the > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> following > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> configuration > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> properties must be added to hbase-site.xml: > >>>>>>>>> > >>>>>>>>>> hbase.backup.enable=true > >>>>>>>>>>>>>>>>>> hbase.master.logcleaner.plugins=YOUR_PLUGINS,org. > >>>>>>>>>>>>>>>>>> apache.hadoop.hbase.backup.master.BackupLogCleaner > >>>>>>>>>>>>>>>>>> hbase.procedure.master.classes=YOUR_CLASSES,org. > >>>>>>>>>>>>>>>>>> apache.hadoop.hbase.backup.master. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> LogRollMasterProcedureManager > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> hbase.procedure.regionserver.classes=YOUR_CLASSES,org. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> apache.hadoop.hbase.backup.regionserver. > >>>>>>>>>>> > >>>>>>>>>>>> LogRollRegionServerProcedureMa > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> nager > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> I would like to thank IBM team and Jerry He for original > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> work, > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Enis, Ted, Stack, Matteo, Jerry for time spent on code > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> reviews > >>>>>>>>>> > >>>>>>>>>>> Special thanks to Ted Yu for his co-development work. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> References: > >>>>>>>>>> > >>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-7912 > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> (original > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> IBM, > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> contains > >>>>>>>>> > >>>>>>>>>> design doc) > >>>>>>>>>>> > >>>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-14030 (Phase > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> 1) > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-14123 (Phase > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> 2) > >>>>>>>>> > >>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-14414 (Phase > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> 3) > >>>>>>>>> > >>>>>>>>>> Please vote +1/-1 by midnight Pacific Time (00:00 > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> -0800 GMT) on March 11th on whether or not we should > >>>>>>>>> > >>>>>>>>>> merge > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> this > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> into > >>>>>>>>> > >>>>>>>>>> the > >>>>>>>>>>>>> > >>>>>>>>>>>>>> current master. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> -Vladimir Rodionov > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> -- > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Best regards, > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> - Andy > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> If you are given a choice, you believe you have acted > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> freely. - > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Raymond > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> Teller (via Peter Watts) > >>>>>>>>>> > >>>>>>>>>>> -- > >>>>>>>>>>>>>>> Best regards, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> - Andy > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> If you are given a choice, you believe you have acted > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> freely. - > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> Raymond > >>>>>>>>>>>>> > >>>>>>>>>>>> Teller (via Peter Watts) > >>>>>>>>> > >>>>>>>>>> -- > >>>>>>>>>>>>> Best regards, > >>>>>>>>>>>>> > >>>>>>>>>>>>> - Andy > >>>>>>>>>>>>> > >>>>>>>>>>>>> If you are given a choice, you believe you have acted > freely. - > >>>>>>>>>>>>> > >>>>>>>>>>>>> Raymond > >>>>>>>>>>>>> > >>>>>>>>>>>> Teller (via Peter Watts) > >>>>>>>>>>> > >>>>>>>>>> > >>>>>> > >>>> > >> > >> >
