I have no vote here, but I'd argue that HBASE-14417 and HBASE-14141 shouldn't be blockers. I agree that HBASE-15227 to add fault tolerance is a blocker.
HBASE-14417 is support for incrementally backing up bulk loaded rows. That's an important feature, but if you don't use bulk loads, or don't care about _incremental_ backups of bulk loads, you'd be able to use backup quite happily without it. Even if you bulk load occasionally, you can do a full backup afterward. In the meantime, the docs and help text would just need to make the limitation clear in big bold letters. :-) HBASE-14141 is allowing HBase Backup to filter out unnecessary data from its incremental backups. Since the backup tool allows you to specify that only certain tables be backed up, incremental backups at WAL granularity will accidentally backup some rows from unneeded tables. This doesn't affect the correctness of the to-be-backed-up tables backups or restores, only its storage cost. The feature works, but there's an important storage optimization to be done. As someone who works on clusters that don't use bulk load, and would want to backup all tables, neither of these seems like a showstopper. However, if HBASE-14141 would be a breaking change to the backup format, then that would change my mind about it being a blocker. Geoffrey On Fri, Mar 10, 2017 at 3:01 PM, Andrew Purtell <[email protected]> wrote: > Thanks for the offer but I like that you were honest about compiling a list > of issues that you thought were blockers for release. Since this proposal > is a merge into 2.0, and we are trying to release 2.0, I am -1 on this > merge until those blockers are addressed. > > I had a look at the list. > > I think the documentation issue is important but not actually a blocker. > That may be a controversial opinion, but documentation can be back-filled > worst case. So take HBASE-17133 off the list. > > Remaining are effectively HBASE-14417, HBASE-14141, and HBASE-15227. They > all have patches attached to the respective JIRAs so completing this work > won't be onerous. Get these committed and I will lift my -1. The others who > voted +1 on this thread surely can help with that. > > Thanks. > > > On Fri, Mar 10, 2017 at 2:32 PM, Vladimir Rodionov <[email protected] > > > wrote: > > > No problem I will downgrade Blockers to Majors if it scares you, Andrew > 🙂 > > > > Sent from my iPhone > > > > > On Mar 10, 2017, at 1:52 PM, Andrew Purtell <[email protected]> > wrote: > > > > > > ​I know the merge of this feature has lagged substantially. I think > that > > is > > > regrettable but on another thread we are lamenting that 2.0 is already > > > late. Unless I misunderstand, this is a proposal to merge something > with > > > known blockers into trunk before we branch it for 2.0 which will > > > effectively prevent that release because these blockers will be there. > I > > am > > > inclined to veto. Probably we should not propose branch merges into > code > > we > > > are trying to get out the door with known blockers. Why not do that > work > > > first? It seems an obvious question. Perhaps I am missing something. > > > > > > If we can branch for 2.0 now and then merge this, and not into the 2.0 > > > branch, I would vote +1 for branch merge even with known blockers > > pending. > > > ​ > > > > > > On Fri, Mar 10, 2017 at 1:42 PM, Vladimir Rodionov < > > [email protected]> > > > wrote: > > > > > >> They are not blockers for merge - only for 2.0. GA > > >> As I said already the feature is usable right now > > >> We would like to continue working on master and we would like to see a > > >> commitment from community > > >> > > >> Sent from my iPhone > > >> > > >> On Mar 10, 2017, at 11:16 AM, Andrew Purtell <[email protected]> > > wrote: > > >> > > >>>> Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release. > > >>> > > >>> If we have identified blockers, why merge this before they are in? > > >>> Otherwise we can't release 2.0, and it is overdue. > > >>> > > >>> > > >>> On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov < > > >> [email protected]> > > >>> wrote: > > >>> > > >>>> Hello, HBase folks > > >>>> > > >>>> For your consideration today is Backup/Restore feature for Apache > > HBAse > > >>>> 2.0. > > >>>> Backup code is available as a mega patch in HBASE-14123 (v61), > applies > > >>>> cleanly to the current master, all test PASS, patch has no other > > issues. > > >>>> > > >>>> The patch has gone through numerous rounds of code reviews and has > > >> probably > > >>>> the most lengthy discussion thread on Apache JIRA (HBASE-14123) :) > > >>>> > > >>>> The work has been split into 3 phases (HBASE-14030, 14123, 14414) > Two > > >> first > > >>>> are complete, third one is still in progress. > > >>>> > > >>>> > > >>>> *** Summary of work HBASE-14123 > > >>>> > > >>>> The new feature introduces new command-line extensions to the hbase > > >> command > > >>>> and, from the client side, is accessible through command-line only > > >>>> Operations: > > >>>> * Create full backup on a list of tables or backup set > > >>>> * Create incremental backup image for table list or backup set > > >>>> * Restore list of tables from a given backup image > > >>>> * Show current backup progress > > >>>> * Delete backup image and all related images > > >>>> * Show history of backups > > >>>> * Backup set operations: create backup set, add/remove table to/from > > >> backup > > >>>> set, etc > > >>>> > > >>>> In the current implementation, the feature is already usable, > meaning > > >> that > > >>>> users can backup tables and restore them using provided command-line > > >> tools. > > >>>> Both: full and incremental backups are supported. > > >>>> This work is based on original work of IBM team (HBASE-7912). The > full > > >> list > > >>>> of JIRAs included in this mega patch can be found in three umbrella > > >> JIRAs: > > >>>> HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 (Phase > 3 > > - > > >> all > > >>>> resolved ones made it into the patch) > > >>>> > > >>>> *** What are the remaining work items > > >>>> > > >>>> All remaining items can be found in Phase 3 umbrella JIRA: > > HBASE-14414. > > >>>> They are split into 3 groups: BLOCKER, CRITICAL, MAJOR > > >>>> Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release. > > >>>> > > >>>> ***** BLOCKER > > >>>> > > >>>> * HBASE-14417 Incremental backup and bulk loading ( Patch available) > > >>>> * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images > > >>>> * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup to > > >>>> include only edits from backup tables (Patch available) > > >>>> * HBASE-17133 Backup documentation > > >>>> * HBASE-15227 Fault tolerance support > > >>>> > > >>>> ***** CRITICAL > > >>>> > > >>>> * HBASE-16465 Disable split/merges during backup > > >>>> > > >>>> We have umbrella JIRA (HBASE-14414) to track all the remaining work > > >>>> All the BLOCKER and CRITICAL JIRAs currently in open state will be > > >>>> implemented by 2.0 release time. Some MAJOR too, but it depends on > > >> resource > > >>>> availability > > >>>> The former development branch (HBASE-7912) is obsolete and will be > > >>>> closed/deleted after the merge. > > >>>> We want backup to be a GA feature in 2.0 > > >>>> We are going to support full backward compatibility for backup tool > in > > >> 2.0 > > >>>> and onwards. > > >>>> > > >>>> **** Configuration > > >>>> > > >>>> Backup is disabled, by default. To enable it, the following > > >> configuration > > >>>> properties must be added to hbase-site.xml: > > >>>> > > >>>> hbase.backup.enable=true > > >>>> hbase.master.logcleaner.plugins=YOUR_PLUGINS,org. > > >>>> apache.hadoop.hbase.backup.master.BackupLogCleaner > > >>>> hbase.procedure.master.classes=YOUR_CLASSES,org. > > >>>> apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager > > >>>> hbase.procedure.regionserver.classes=YOUR_CLASSES,org. > > >>>> apache.hadoop.hbase.backup.regionserver. > > LogRollRegionServerProcedureMa > > >>>> nager > > >>>> > > >>>> > > >>>> I would like to thank IBM team and Jerry He for original work, > > >>>> > > >>>> Enis, Ted, Stack, Matteo, Jerry for time spent on code reviews > > >>>> > > >>>> Special thanks to Ted Yu for his co-development work. > > >>>> > > >>>> References: > > >>>> > > >>>> https://issues.apache.org/jira/browse/HBASE-7912 (original IBM, > > >> contains > > >>>> design doc) > > >>>> https://issues.apache.org/jira/browse/HBASE-14030 (Phase 1) > > >>>> https://issues.apache.org/jira/browse/HBASE-14123 (Phase 2) > > >>>> https://issues.apache.org/jira/browse/HBASE-14414 (Phase 3) > > >>>> > > >>>> Please vote +1/-1 by midnight Pacific Time (00:00 > > >>>> -0800 GMT) on March 11th ​on whether or not we should merge this > into > > >> the > > >>>> current master. > > >>>> > > >>>> -Vladimir Rodionov > > >>>> > > >>> > > >>> > > >>> > > >>> -- > > >>> Best regards, > > >>> > > >>> - Andy > > >>> > > >>> If you are given a choice, you believe you have acted freely. - > Raymond > > >>> Teller (via Peter Watts) > > >> > > > > > > > > > > > > -- > > > Best regards, > > > > > > - Andy > > > > > > If you are given a choice, you believe you have acted freely. - Raymond > > > Teller (via Peter Watts) > > > > > > -- > Best regards, > > - Andy > > If you are given a choice, you believe you have acted freely. - Raymond > Teller (via Peter Watts) >
