I think there is some misconception of using the term "blockers" for referring to those jiras. My understanding is that those three jiras are blockers for the backup functionality to be more mature and more usable. But they are not release blockers. Let's say we merged the code in, and for some reason those did not get addressed in time. We can still do the 2.0 release without having to wait for the commits. We can instead mark the "backup" feature as experimental with known issues and go on with the release. In that sense they are not real release blockers.
We are proposing the merge at this time because of the above that maintaining this in a branch is becoming extremely costly and not productive for the HBase community. Realistically, we cannot have the luxury of having to wait another couple of months and doing yet another giant round of reviews because the code base is a moving target. Enis On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das <d...@hortonworks.com> wrote: > Vlad, on the first point, I think what Stack is saying is that creating > the new branch (as Ted did) ignores the feedback incorporated thus far in > the iterations of the mega-patch. That's a wrong way to go. > On the separation into a backup module, again, that was reverted to ease > reviews of the mega-patch, and was noted as work to be done later. I think > Stack just wants to make the list of remaining work more complete by citing > that as pending work. > ________________________________________ > From: Vladimir Rodionov <vladrodio...@gmail.com> > Sent: Monday, March 13, 2017 3:09 PM > To: dev@hbase.apache.org > Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing > 3/11/2017 > > >> It ignores the feedback > > If I "ignore" feedback, I put my comment - why? I am always open for > further discussions. If reviewer does not insist on a particular request - > it will be dropped. I think it is fair. > > >> he list is incomplete because a bunch of > >> follow-ons came of the review cycle (including moving backup/restore out > of > >> core to live in its own module). > > For those who were not following our lengthy conversation on a review > board, separation of a backup code into a separate module has been > done last year, but has been reverted back by request of a reviewer. > > > -Vladimir > > On Mon, Mar 13, 2017 at 2:23 PM, Stack <st...@duboce.net> wrote: > > > On Fri, Mar 10, 2017 at 9:09 PM, Stack <st...@duboce.net> wrote: > > > > > On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu <yuzhih...@gmail.com> wrote: > > > > > >> HBASE-14123 branch has been created, with Vlad's mega patch v61. > > >> > > > > > > The patch put up for VOTE here was done on a branch. The call to merge > > > seems to have been premature given the many cycles of review and test > > that > > > happened subsequent (The cycles burned a bunch of dev resource). > > > > > > The patch as is is now in a state where it is too big for our infra; rb > > > and JIRA are creaking under the size and # of iterations. > > > > > > Adding finish of new JIRAs to this merge implies a new round of review > > and > > > test of an already massive patch. Who is going to do this work? > > > > > > Going back to a new branch seems wrong route to take. > > > > > > St.Ack > > > > > > > > > > > To be more explicit, this patch was developed on a branch and then a > bunch > > of dev resources were burned getting it into a state where it could be > > merged to master. Going back to a branch to bulk up the merge so it > > includes more JIRAs than the many it already incorporates is the wrong > > direction for us to be headed in. It ignores the feedback given and the > > work done by Vladimir slimming down an already over-broad scope. It is > also > > predicated on abundant review and testing resource being on tap to cycle > on > > a feature that is useful, but non-core. > > > > The patch is ready for merge IMO. Geoffrey makes a nice list of what is > > still to do though IIRC, the list is incomplete because a bunch of > > follow-ons came of the review cycle (including moving backup/restore out > of > > core to live in its own module). > > > > The patch needs three votes to merge. I am not against merge but I am not > > voting for the patch because I do have any more time to spend on this > > non-core feature and feel that a vote will have me assume a > responsibility > > I will not shirk. > > > > S > > > > > > > > > > > > > > > > > >> FYI > > >> > > >> On Fri, Mar 10, 2017 at 3:30 PM, Ted Yu <yuzhih...@gmail.com> wrote: > > >> > > >> > Thanks for the feedback, Andrew. > > >> > > > >> > How about the following plan: > > >> > > > >> > create branch HBASE-14123 off of master with mega patch v61 as the > > first > > >> > commit (reviewed by Stack and Enis) > > >> > Vlad and I continue development (the 3 blockers) based on > HBASE-14123 > > >> > branch > > >> > when all of the blockers get +1 and merged into HBASE-14123 branch, > we > > >> > propose to community for merging into branch-2 (master branch, if > > >> branch-2 > > >> > doesn't materialize for whatever reason) again > > >> > > > >> > Cheers > > >> > > > >> > > > >> > On Fri, Mar 10, 2017 at 3:01 PM, Andrew Purtell < > apurt...@apache.org> > > >> > wrote: > > >> > > > >> >> Thanks for the offer but I like that you were honest about > compiling > > a > > >> >> list > > >> >> of issues that you thought were blockers for release. Since this > > >> proposal > > >> >> is a merge into 2.0, and we are trying to release 2.0, I am -1 on > > this > > >> >> merge until those blockers are addressed. > > >> >> > > >> >> I had a look at the list. > > >> >> > > >> >> I think the documentation issue is important but not actually a > > >> blocker. > > >> >> That may be a controversial opinion, but documentation can be > > >> back-filled > > >> >> worst case. So take HBASE-17133 off the list. > > >> >> > > >> >> Remaining are effectively HBASE-14417, HBASE-14141, and > HBASE-15227. > > >> They > > >> >> all have patches attached to the respective JIRAs so completing > this > > >> work > > >> >> won't be onerous. Get these committed and I will lift my -1. The > > others > > >> >> who > > >> >> voted +1 on this thread surely can help with that. > > >> >> > > >> >> Thanks. > > >> >> > > >> >> > > >> >> On Fri, Mar 10, 2017 at 2:32 PM, Vladimir Rodionov < > > >> >> vladrodio...@gmail.com> > > >> >> wrote: > > >> >> > > >> >> > No problem I will downgrade Blockers to Majors if it scares you, > > >> Andrew > > >> >> 🙂 > > >> >> > > > >> >> > Sent from my iPhone > > >> >> > > > >> >> > > On Mar 10, 2017, at 1:52 PM, Andrew Purtell < > apurt...@apache.org > > > > > >> >> wrote: > > >> >> > > > > >> >> > > I know the merge of this feature has lagged substantially. I > > think > > >> >> that > > >> >> > is > > >> >> > > regrettable but on another thread we are lamenting that 2.0 is > > >> already > > >> >> > > late. Unless I misunderstand, this is a proposal to merge > > something > > >> >> with > > >> >> > > known blockers into trunk before we branch it for 2.0 which > will > > >> >> > > effectively prevent that release because these blockers will be > > >> >> there. I > > >> >> > am > > >> >> > > inclined to veto. Probably we should not propose branch merges > > into > > >> >> code > > >> >> > we > > >> >> > > are trying to get out the door with known blockers. Why not do > > that > > >> >> work > > >> >> > > first? It seems an obvious question. Perhaps I am missing > > >> something. > > >> >> > > > > >> >> > > If we can branch for 2.0 now and then merge this, and not into > > the > > >> 2.0 > > >> >> > > branch, I would vote +1 for branch merge even with known > blockers > > >> >> > pending. > > >> >> > > > > >> >> > > > > >> >> > > On Fri, Mar 10, 2017 at 1:42 PM, Vladimir Rodionov < > > >> >> > vladrodio...@gmail.com> > > >> >> > > wrote: > > >> >> > > > > >> >> > >> They are not blockers for merge - only for 2.0. GA > > >> >> > >> As I said already the feature is usable right now > > >> >> > >> We would like to continue working on master and we would like > to > > >> see > > >> >> a > > >> >> > >> commitment from community > > >> >> > >> > > >> >> > >> Sent from my iPhone > > >> >> > >> > > >> >> > >> On Mar 10, 2017, at 11:16 AM, Andrew Purtell < > > apurt...@apache.org > > >> > > > >> >> > wrote: > > >> >> > >> > > >> >> > >>>> Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 > > >> release. > > >> >> > >>> > > >> >> > >>> If we have identified blockers, why merge this before they > are > > >> in? > > >> >> > >>> Otherwise we can't release 2.0, and it is overdue. > > >> >> > >>> > > >> >> > >>> > > >> >> > >>> On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov < > > >> >> > >> vladrodio...@gmail.com> > > >> >> > >>> wrote: > > >> >> > >>> > > >> >> > >>>> Hello, HBase folks > > >> >> > >>>> > > >> >> > >>>> For your consideration today is Backup/Restore feature for > > >> Apache > > >> >> > HBAse > > >> >> > >>>> 2.0. > > >> >> > >>>> Backup code is available as a mega patch in HBASE-14123 > (v61), > > >> >> applies > > >> >> > >>>> cleanly to the current master, all test PASS, patch has no > > other > > >> >> > issues. > > >> >> > >>>> > > >> >> > >>>> The patch has gone through numerous rounds of code reviews > and > > >> has > > >> >> > >> probably > > >> >> > >>>> the most lengthy discussion thread on Apache JIRA > > (HBASE-14123) > > >> :) > > >> >> > >>>> > > >> >> > >>>> The work has been split into 3 phases (HBASE-14030, 14123, > > >> 14414) > > >> >> Two > > >> >> > >> first > > >> >> > >>>> are complete, third one is still in progress. > > >> >> > >>>> > > >> >> > >>>> > > >> >> > >>>> *** Summary of work HBASE-14123 > > >> >> > >>>> > > >> >> > >>>> The new feature introduces new command-line extensions to > the > > >> hbase > > >> >> > >> command > > >> >> > >>>> and, from the client side, is accessible through > command-line > > >> only > > >> >> > >>>> Operations: > > >> >> > >>>> * Create full backup on a list of tables or backup set > > >> >> > >>>> * Create incremental backup image for table list or backup > set > > >> >> > >>>> * Restore list of tables from a given backup image > > >> >> > >>>> * Show current backup progress > > >> >> > >>>> * Delete backup image and all related images > > >> >> > >>>> * Show history of backups > > >> >> > >>>> * Backup set operations: create backup set, add/remove table > > >> >> to/from > > >> >> > >> backup > > >> >> > >>>> set, etc > > >> >> > >>>> > > >> >> > >>>> In the current implementation, the feature is already > usable, > > >> >> meaning > > >> >> > >> that > > >> >> > >>>> users can backup tables and restore them using provided > > >> >> command-line > > >> >> > >> tools. > > >> >> > >>>> Both: full and incremental backups are supported. > > >> >> > >>>> This work is based on original work of IBM team > (HBASE-7912). > > >> The > > >> >> full > > >> >> > >> list > > >> >> > >>>> of JIRAs included in this mega patch can be found in three > > >> umbrella > > >> >> > >> JIRAs: > > >> >> > >>>> HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 > > >> >> (Phase 3 > > >> >> > - > > >> >> > >> all > > >> >> > >>>> resolved ones made it into the patch) > > >> >> > >>>> > > >> >> > >>>> *** What are the remaining work items > > >> >> > >>>> > > >> >> > >>>> All remaining items can be found in Phase 3 umbrella JIRA: > > >> >> > HBASE-14414. > > >> >> > >>>> They are split into 3 groups: BLOCKER, CRITICAL, MAJOR > > >> >> > >>>> Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 > > >> release. > > >> >> > >>>> > > >> >> > >>>> ***** BLOCKER > > >> >> > >>>> > > >> >> > >>>> * HBASE-14417 Incremental backup and bulk loading ( Patch > > >> >> available) > > >> >> > >>>> * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup > > images > > >> >> > >>>> * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on > > >> backup > > >> >> to > > >> >> > >>>> include only edits from backup tables (Patch available) > > >> >> > >>>> * HBASE-17133 Backup documentation > > >> >> > >>>> * HBASE-15227 Fault tolerance support > > >> >> > >>>> > > >> >> > >>>> ***** CRITICAL > > >> >> > >>>> > > >> >> > >>>> * HBASE-16465 Disable split/merges during backup > > >> >> > >>>> > > >> >> > >>>> We have umbrella JIRA (HBASE-14414) to track all the > remaining > > >> work > > >> >> > >>>> All the BLOCKER and CRITICAL JIRAs currently in open state > > will > > >> be > > >> >> > >>>> implemented by 2.0 release time. Some MAJOR too, but it > > depends > > >> on > > >> >> > >> resource > > >> >> > >>>> availability > > >> >> > >>>> The former development branch (HBASE-7912) is obsolete and > > will > > >> be > > >> >> > >>>> closed/deleted after the merge. > > >> >> > >>>> We want backup to be a GA feature in 2.0 > > >> >> > >>>> We are going to support full backward compatibility for > backup > > >> >> tool in > > >> >> > >> 2.0 > > >> >> > >>>> and onwards. > > >> >> > >>>> > > >> >> > >>>> **** Configuration > > >> >> > >>>> > > >> >> > >>>> Backup is disabled, by default. To enable it, the following > > >> >> > >> configuration > > >> >> > >>>> properties must be added to hbase-site.xml: > > >> >> > >>>> > > >> >> > >>>> hbase.backup.enable=true > > >> >> > >>>> hbase.master.logcleaner.plugins=YOUR_PLUGINS,org. > > >> >> > >>>> apache.hadoop.hbase.backup.master.BackupLogCleaner > > >> >> > >>>> hbase.procedure.master.classes=YOUR_CLASSES,org. > > >> >> > >>>> apache.hadoop.hbase.backup.master. > > LogRollMasterProcedureManager > > >> >> > >>>> hbase.procedure.regionserver.classes=YOUR_CLASSES,org. > > >> >> > >>>> apache.hadoop.hbase.backup.regionserver. > > >> >> > LogRollRegionServerProcedureMa > > >> >> > >>>> nager > > >> >> > >>>> > > >> >> > >>>> > > >> >> > >>>> I would like to thank IBM team and Jerry He for original > work, > > >> >> > >>>> > > >> >> > >>>> Enis, Ted, Stack, Matteo, Jerry for time spent on code > reviews > > >> >> > >>>> > > >> >> > >>>> Special thanks to Ted Yu for his co-development work. > > >> >> > >>>> > > >> >> > >>>> References: > > >> >> > >>>> > > >> >> > >>>> https://issues.apache.org/jira/browse/HBASE-7912 (original > > IBM, > > >> >> > >> contains > > >> >> > >>>> design doc) > > >> >> > >>>> https://issues.apache.org/jira/browse/HBASE-14030 (Phase 1) > > >> >> > >>>> https://issues.apache.org/jira/browse/HBASE-14123 (Phase 2) > > >> >> > >>>> https://issues.apache.org/jira/browse/HBASE-14414 (Phase 3) > > >> >> > >>>> > > >> >> > >>>> Please vote +1/-1 by midnight Pacific Time (00:00 > > >> >> > >>>> -0800 GMT) on March 11th on whether or not we should merge > > >> this > > >> >> into > > >> >> > >> the > > >> >> > >>>> current master. > > >> >> > >>>> > > >> >> > >>>> -Vladimir Rodionov > > >> >> > >>>> > > >> >> > >>> > > >> >> > >>> > > >> >> > >>> > > >> >> > >>> -- > > >> >> > >>> Best regards, > > >> >> > >>> > > >> >> > >>> - Andy > > >> >> > >>> > > >> >> > >>> If you are given a choice, you believe you have acted > freely. - > > >> >> Raymond > > >> >> > >>> Teller (via Peter Watts) > > >> >> > >> > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > -- > > >> >> > > Best regards, > > >> >> > > > > >> >> > > - Andy > > >> >> > > > > >> >> > > If you are given a choice, you believe you have acted freely. - > > >> >> Raymond > > >> >> > > Teller (via Peter Watts) > > >> >> > > > >> >> > > >> >> > > >> >> > > >> >> -- > > >> >> Best regards, > > >> >> > > >> >> - Andy > > >> >> > > >> >> If you are given a choice, you believe you have acted freely. - > > Raymond > > >> >> Teller (via Peter Watts) > > >> >> > > >> > > > >> > > > >> > > > > > > > > >
