On Fri, Mar 10, 2017 at 9:09 PM, Stack <st...@duboce.net> wrote: > On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu <yuzhih...@gmail.com> wrote: > >> HBASE-14123 branch has been created, with Vlad's mega patch v61. >> > > The patch put up for VOTE here was done on a branch. The call to merge > seems to have been premature given the many cycles of review and test that > happened subsequent (The cycles burned a bunch of dev resource). > > The patch as is is now in a state where it is too big for our infra; rb > and JIRA are creaking under the size and # of iterations. > > Adding finish of new JIRAs to this merge implies a new round of review and > test of an already massive patch. Who is going to do this work? > > Going back to a new branch seems wrong route to take. > > St.Ack > > > To be more explicit, this patch was developed on a branch and then a bunch of dev resources were burned getting it into a state where it could be merged to master. Going back to a branch to bulk up the merge so it includes more JIRAs than the many it already incorporates is the wrong direction for us to be headed in. It ignores the feedback given and the work done by Vladimir slimming down an already over-broad scope. It is also predicated on abundant review and testing resource being on tap to cycle on a feature that is useful, but non-core.
The patch is ready for merge IMO. Geoffrey makes a nice list of what is still to do though IIRC, the list is incomplete because a bunch of follow-ons came of the review cycle (including moving backup/restore out of core to live in its own module). The patch needs three votes to merge. I am not against merge but I am not voting for the patch because I do have any more time to spend on this non-core feature and feel that a vote will have me assume a responsibility I will not shirk. S > > > >> FYI >> >> On Fri, Mar 10, 2017 at 3:30 PM, Ted Yu <yuzhih...@gmail.com> wrote: >> >> > Thanks for the feedback, Andrew. >> > >> > How about the following plan: >> > >> > create branch HBASE-14123 off of master with mega patch v61 as the first >> > commit (reviewed by Stack and Enis) >> > Vlad and I continue development (the 3 blockers) based on HBASE-14123 >> > branch >> > when all of the blockers get +1 and merged into HBASE-14123 branch, we >> > propose to community for merging into branch-2 (master branch, if >> branch-2 >> > doesn't materialize for whatever reason) again >> > >> > Cheers >> > >> > >> > On Fri, Mar 10, 2017 at 3:01 PM, Andrew Purtell <apurt...@apache.org> >> > wrote: >> > >> >> Thanks for the offer but I like that you were honest about compiling a >> >> list >> >> of issues that you thought were blockers for release. Since this >> proposal >> >> is a merge into 2.0, and we are trying to release 2.0, I am -1 on this >> >> merge until those blockers are addressed. >> >> >> >> I had a look at the list. >> >> >> >> I think the documentation issue is important but not actually a >> blocker. >> >> That may be a controversial opinion, but documentation can be >> back-filled >> >> worst case. So take HBASE-17133 off the list. >> >> >> >> Remaining are effectively HBASE-14417, HBASE-14141, and HBASE-15227. >> They >> >> all have patches attached to the respective JIRAs so completing this >> work >> >> won't be onerous. Get these committed and I will lift my -1. The others >> >> who >> >> voted +1 on this thread surely can help with that. >> >> >> >> Thanks. >> >> >> >> >> >> On Fri, Mar 10, 2017 at 2:32 PM, Vladimir Rodionov < >> >> vladrodio...@gmail.com> >> >> wrote: >> >> >> >> > No problem I will downgrade Blockers to Majors if it scares you, >> Andrew >> >> 🙂 >> >> > >> >> > Sent from my iPhone >> >> > >> >> > > On Mar 10, 2017, at 1:52 PM, Andrew Purtell <apurt...@apache.org> >> >> wrote: >> >> > > >> >> > > ​I know the merge of this feature has lagged substantially. I think >> >> that >> >> > is >> >> > > regrettable but on another thread we are lamenting that 2.0 is >> already >> >> > > late. Unless I misunderstand, this is a proposal to merge something >> >> with >> >> > > known blockers into trunk before we branch it for 2.0 which will >> >> > > effectively prevent that release because these blockers will be >> >> there. I >> >> > am >> >> > > inclined to veto. Probably we should not propose branch merges into >> >> code >> >> > we >> >> > > are trying to get out the door with known blockers. Why not do that >> >> work >> >> > > first? It seems an obvious question. Perhaps I am missing >> something. >> >> > > >> >> > > If we can branch for 2.0 now and then merge this, and not into the >> 2.0 >> >> > > branch, I would vote +1 for branch merge even with known blockers >> >> > pending. >> >> > > ​ >> >> > > >> >> > > On Fri, Mar 10, 2017 at 1:42 PM, Vladimir Rodionov < >> >> > vladrodio...@gmail.com> >> >> > > wrote: >> >> > > >> >> > >> They are not blockers for merge - only for 2.0. GA >> >> > >> As I said already the feature is usable right now >> >> > >> We would like to continue working on master and we would like to >> see >> >> a >> >> > >> commitment from community >> >> > >> >> >> > >> Sent from my iPhone >> >> > >> >> >> > >> On Mar 10, 2017, at 11:16 AM, Andrew Purtell <apurt...@apache.org >> > >> >> > wrote: >> >> > >> >> >> > >>>> Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 >> release. >> >> > >>> >> >> > >>> If we have identified blockers, why merge this before they are >> in? >> >> > >>> Otherwise we can't release 2.0, and it is overdue. >> >> > >>> >> >> > >>> >> >> > >>> On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov < >> >> > >> vladrodio...@gmail.com> >> >> > >>> wrote: >> >> > >>> >> >> > >>>> Hello, HBase folks >> >> > >>>> >> >> > >>>> For your consideration today is Backup/Restore feature for >> Apache >> >> > HBAse >> >> > >>>> 2.0. >> >> > >>>> Backup code is available as a mega patch in HBASE-14123 (v61), >> >> applies >> >> > >>>> cleanly to the current master, all test PASS, patch has no other >> >> > issues. >> >> > >>>> >> >> > >>>> The patch has gone through numerous rounds of code reviews and >> has >> >> > >> probably >> >> > >>>> the most lengthy discussion thread on Apache JIRA (HBASE-14123) >> :) >> >> > >>>> >> >> > >>>> The work has been split into 3 phases (HBASE-14030, 14123, >> 14414) >> >> Two >> >> > >> first >> >> > >>>> are complete, third one is still in progress. >> >> > >>>> >> >> > >>>> >> >> > >>>> *** Summary of work HBASE-14123 >> >> > >>>> >> >> > >>>> The new feature introduces new command-line extensions to the >> hbase >> >> > >> command >> >> > >>>> and, from the client side, is accessible through command-line >> only >> >> > >>>> Operations: >> >> > >>>> * Create full backup on a list of tables or backup set >> >> > >>>> * Create incremental backup image for table list or backup set >> >> > >>>> * Restore list of tables from a given backup image >> >> > >>>> * Show current backup progress >> >> > >>>> * Delete backup image and all related images >> >> > >>>> * Show history of backups >> >> > >>>> * Backup set operations: create backup set, add/remove table >> >> to/from >> >> > >> backup >> >> > >>>> set, etc >> >> > >>>> >> >> > >>>> In the current implementation, the feature is already usable, >> >> meaning >> >> > >> that >> >> > >>>> users can backup tables and restore them using provided >> >> command-line >> >> > >> tools. >> >> > >>>> Both: full and incremental backups are supported. >> >> > >>>> This work is based on original work of IBM team (HBASE-7912). >> The >> >> full >> >> > >> list >> >> > >>>> of JIRAs included in this mega patch can be found in three >> umbrella >> >> > >> JIRAs: >> >> > >>>> HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 >> >> (Phase 3 >> >> > - >> >> > >> all >> >> > >>>> resolved ones made it into the patch) >> >> > >>>> >> >> > >>>> *** What are the remaining work items >> >> > >>>> >> >> > >>>> All remaining items can be found in Phase 3 umbrella JIRA: >> >> > HBASE-14414. >> >> > >>>> They are split into 3 groups: BLOCKER, CRITICAL, MAJOR >> >> > >>>> Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 >> release. >> >> > >>>> >> >> > >>>> ***** BLOCKER >> >> > >>>> >> >> > >>>> * HBASE-14417 Incremental backup and bulk loading ( Patch >> >> available) >> >> > >>>> * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images >> >> > >>>> * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on >> backup >> >> to >> >> > >>>> include only edits from backup tables (Patch available) >> >> > >>>> * HBASE-17133 Backup documentation >> >> > >>>> * HBASE-15227 Fault tolerance support >> >> > >>>> >> >> > >>>> ***** CRITICAL >> >> > >>>> >> >> > >>>> * HBASE-16465 Disable split/merges during backup >> >> > >>>> >> >> > >>>> We have umbrella JIRA (HBASE-14414) to track all the remaining >> work >> >> > >>>> All the BLOCKER and CRITICAL JIRAs currently in open state will >> be >> >> > >>>> implemented by 2.0 release time. Some MAJOR too, but it depends >> on >> >> > >> resource >> >> > >>>> availability >> >> > >>>> The former development branch (HBASE-7912) is obsolete and will >> be >> >> > >>>> closed/deleted after the merge. >> >> > >>>> We want backup to be a GA feature in 2.0 >> >> > >>>> We are going to support full backward compatibility for backup >> >> tool in >> >> > >> 2.0 >> >> > >>>> and onwards. >> >> > >>>> >> >> > >>>> **** Configuration >> >> > >>>> >> >> > >>>> Backup is disabled, by default. To enable it, the following >> >> > >> configuration >> >> > >>>> properties must be added to hbase-site.xml: >> >> > >>>> >> >> > >>>> hbase.backup.enable=true >> >> > >>>> hbase.master.logcleaner.plugins=YOUR_PLUGINS,org. >> >> > >>>> apache.hadoop.hbase.backup.master.BackupLogCleaner >> >> > >>>> hbase.procedure.master.classes=YOUR_CLASSES,org. >> >> > >>>> apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager >> >> > >>>> hbase.procedure.regionserver.classes=YOUR_CLASSES,org. >> >> > >>>> apache.hadoop.hbase.backup.regionserver. >> >> > LogRollRegionServerProcedureMa >> >> > >>>> nager >> >> > >>>> >> >> > >>>> >> >> > >>>> I would like to thank IBM team and Jerry He for original work, >> >> > >>>> >> >> > >>>> Enis, Ted, Stack, Matteo, Jerry for time spent on code reviews >> >> > >>>> >> >> > >>>> Special thanks to Ted Yu for his co-development work. >> >> > >>>> >> >> > >>>> References: >> >> > >>>> >> >> > >>>> https://issues.apache.org/jira/browse/HBASE-7912 (original IBM, >> >> > >> contains >> >> > >>>> design doc) >> >> > >>>> https://issues.apache.org/jira/browse/HBASE-14030 (Phase 1) >> >> > >>>> https://issues.apache.org/jira/browse/HBASE-14123 (Phase 2) >> >> > >>>> https://issues.apache.org/jira/browse/HBASE-14414 (Phase 3) >> >> > >>>> >> >> > >>>> Please vote +1/-1 by midnight Pacific Time (00:00 >> >> > >>>> -0800 GMT) on March 11th ​on whether or not we should merge >> this >> >> into >> >> > >> the >> >> > >>>> current master. >> >> > >>>> >> >> > >>>> -Vladimir Rodionov >> >> > >>>> >> >> > >>> >> >> > >>> >> >> > >>> >> >> > >>> -- >> >> > >>> Best regards, >> >> > >>> >> >> > >>> - Andy >> >> > >>> >> >> > >>> If you are given a choice, you believe you have acted freely. - >> >> Raymond >> >> > >>> Teller (via Peter Watts) >> >> > >> >> >> > > >> >> > > >> >> > > >> >> > > -- >> >> > > Best regards, >> >> > > >> >> > > - Andy >> >> > > >> >> > > If you are given a choice, you believe you have acted freely. - >> >> Raymond >> >> > > Teller (via Peter Watts) >> >> > >> >> >> >> >> >> >> >> -- >> >> Best regards, >> >> >> >> - Andy >> >> >> >> If you are given a choice, you believe you have acted freely. - Raymond >> >> Teller (via Peter Watts) >> >> >> > >> > >> > >
