Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
de out for people to use (and to free up our > strained > > > dev > > > > > > >>>>> resources), but what good is some feature if the docs are > > > > > > >>>>> missing/incomplete? > > > > > > >>>>> > > > > > > >>>>> I think I could stomach the docs being inaccurate (with a > > clear > > > > > > >>>>> disclaimer > > > > > > >>>>> that the chapter is incomplete -- that's a 5min task). > But, I > > > > > think I > > > > > > >>>>> need > > > > > > >>>>> an answer about how the feature handles our common dist-sys > > > > > category > > > > > > of > > > > > > >>>>> problems before I can consider whether I'm ok with the > > feature > > > > > > hitting > > > > > > >>>>> 2.0... > > > > > > >>>>> > > > > > > >>>>> I'll also try to throw up a few nodes and play with it to > > > address > > > > > the > > > > > > >>>>> problem as an (ignorant) user ;) > > > > > > >>>>> > > > > > > >>>>> > > > > > > >>>>> Andrew Purtell wrote: > > > > > > >>>>> > > > > > > >>>>> I don't like that issues were identified as "blockers" but > > now > > > > > there > > > > > > is > > > > > > >>>>> > > > > > > >>>>>> an > > > > > > >>>>>> attempt to walk that back. > > > > > > >>>>>> > > > > > > >>>>>> I don't like that development of this feature has lingered > > > for a > > > > > > long > > > > > > >>>>>> time > > > > > > >>>>>> in this unfinished state when this work could have been > done > > > by > > > > > now, > > > > > > >>>>>> now > > > > > > >>>>>> that we are trying to get a 2.0 out the door. Because this > > is > > > a > > > > > > >>>>>> volunteer > > > > > > >>>>>> project I cannot make any demand that it should be done, > > but I > > > > can > > > > > > >>>>>> certainly look at the current state and be nonplussed. > This > > > will > > > > > be > > > > > > >>>>>> yet > > > > > > >>>>>> another half finished thing in 2.0 when this merge > happens. > > > > > Promises > > > > > > >>>>>> to > > > > > > >>>>>> finish the unfinished work are nice but not currency. > > Commits > > > > are > > > > > > >>>>>> currency. > > > > > > >>>>>> I hope at least the fault tolerance changes can be > completed > > > and > > > > > > >>>>>> committed > > > > > > >>>>>> before we spin a 2.0 RC, and without causing a 2.0 release > > to > > > > slip > > > > > > >>>>>> further. > > > > > > >>>>>> > > > > > > >>>>>> Also, marking something experimental should be done on the > > > > merits > > > > > of > > > > > > >>>>>> that > > > > > > >>>>>> evaluation, not simply to justify dropping unfinished work > > > into > > > > a > > > > > > >>>>>> release > > > > > > >>>>>> branch. > > > > > > >>>>>> > > > > > > >>>>>> I will change my vote to -0. > > > > > > >>>>>> > > > > > > >>>>>> > > > > > > >>>>>> On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar< > > > e...@apache.org> > > > > > > >>>>>
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
t; > > > >>>> > > > > > >>>> Once this work is merged in, how is HBASE-15227 not a blocker > > for > > > > 2.0? > > > > > >>>> Because Vlad offered to reduce its severity to make me feel > > > better? > > > > > >>>> Currently the description on the issue is "System must be > > tolerant > > > > to > > > > > >>>> faults. Backup operations MUST be atomic (no partial > completion > > > > state > > > > > in > > > > > >>>> system table)" Sounds like a blocker to me, indeed. It is an > > > honest > > > > > >>>> assessment and I don't think anyone is doing the community a > > favor > > > > by > > > > > >>>> trying to walk that back. > > > > > >>>> > > > > > >>>> > > > > > >>>> On Tue, Mar 14, 2017 at 1:57 PM, Josh Elser > > > > > > wrote: > > > > > >>>> > > > > > >>>> I took a moment to read through the "blockers" as originally > > > > > identified > > > > > >>>> by > > > > > >>>> > > > > > >>>>> Vlad, and (to echo Enis' take) I read the majority of them as > > > being > > > > > >>>>> blockers not for the next release, but for a "full-fledged > > > > feature". > > > > > >>>>> I'm > > > > > >>>>> going to intentional avoid addressing the discussion of > > shipping > > > > > >>>>> partial > > > > > >>>>> features (related, but not relevant at the moment). > > > > > >>>>> > > > > > >>>>> HBASE-15227 is actually the one that bothers me the most, > with > > > > > >>>>> HBASE-17133 > > > > > >>>>> coming in close behind. Vlad, is there any documentation you > > can > > > > > point > > > > > >>>>> me > > > > > >>>>> to about what the current issues are with the current > > > > implementation? > > > > > >>>>> For > > > > > >>>>> example, what happens now if the system has some kind of > > "partial > > > > > >>>>> completion state"? > > > > > >>>>> > > > > > >>>>> Documentation is one of those that is really hard to judge. > We > > > want > > > > > to > > > > > >>>>> get > > > > > >>>>> this code out for people to use (and to free up our strained > > dev > > > > > >>>>> resources), but what good is some feature if the docs are > > > > > >>>>> missing/incomplete? > > > > > >>>>> > > > > > >>>>> I think I could stomach the docs being inaccurate (with a > clear > > > > > >>>>> disclaimer > > > > > >>>>> that the chapter is incomplete -- that's a 5min task). But, I > > > > think I > > > > > >>>>> need > > > > > >>>>> an answer about how the feature handles our common dist-sys > > > > category > > > > > of > > > > > >>>>> problems before I can consider whether I'm ok with the > feature > > > > > hitting > > > > > >>>>> 2.0... > > > > > >>>>> > > > > > >>>>> I'll also try to throw up a few nodes and play with it to > > address > > > > the > > > > > >>>>> problem as an (ignorant) user ;) > > > > > >>>>> > > > > > >>>>> > > > > > >>>>> Andrew Purtell wrote: > > > > > >>>>> > > > > > >>>>> I don't like that issues were identified as "blockers" but > now > > > > there > > > > > is > > > > > >>>>> > > > > > >>>>>> an > > > > > >>>>>>
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
originally > > > > identified > > > > >>>> by > > > > >>>> > > > > >>>>> Vlad, and (to echo Enis' take) I read the majority of them as > > being > > > > >>>>> blockers not for the next release, but for a "full-fledged > > > feature". > > > > >>>>> I'm > > > > >>>>> going to intentional avoid addressing the discussion of > shipping > > > > >>>>> partial > > > > >>>>> features (related, but not relevant at the moment). > > > > >>>>> > > > > >>>>> HBASE-15227 is actually the one that bothers me the most, with > > > > >>>>> HBASE-17133 > > > > >>>>> coming in close behind. Vlad, is there any documentation you > can > > > > point > > > > >>>>> me > > > > >>>>> to about what the current issues are with the current > > > implementation? > > > > >>>>> For > > > > >>>>> example, what happens now if the system has some kind of > "partial > > > > >>>>> completion state"? > > > > >>>>> > > > > >>>>> Documentation is one of those that is really hard to judge. We > > want > > > > to > > > > >>>>> get > > > > >>>>> this code out for people to use (and to free up our strained > dev > > > > >>>>> resources), but what good is some feature if the docs are > > > > >>>>> missing/incomplete? > > > > >>>>> > > > > >>>>> I think I could stomach the docs being inaccurate (with a clear > > > > >>>>> disclaimer > > > > >>>>> that the chapter is incomplete -- that's a 5min task). But, I > > > think I > > > > >>>>> need > > > > >>>>> an answer about how the feature handles our common dist-sys > > > category > > > > of > > > > >>>>> problems before I can consider whether I'm ok with the feature > > > > hitting > > > > >>>>> 2.0... > > > > >>>>> > > > > >>>>> I'll also try to throw up a few nodes and play with it to > address > > > the > > > > >>>>> problem as an (ignorant) user ;) > > > > >>>>> > > > > >>>>> > > > > >>>>> Andrew Purtell wrote: > > > > >>>>> > > > > >>>>> I don't like that issues were identified as "blockers" but now > > > there > > > > is > > > > >>>>> > > > > >>>>>> an > > > > >>>>>> attempt to walk that back. > > > > >>>>>> > > > > >>>>>> I don't like that development of this feature has lingered > for a > > > > long > > > > >>>>>> time > > > > >>>>>> in this unfinished state when this work could have been done > by > > > now, > > > > >>>>>> now > > > > >>>>>> that we are trying to get a 2.0 out the door. Because this is > a > > > > >>>>>> volunteer > > > > >>>>>> project I cannot make any demand that it should be done, but I > > can > > > > >>>>>> certainly look at the current state and be nonplussed. This > will > > > be > > > > >>>>>> yet > > > > >>>>>> another half finished thing in 2.0 when this merge happens. > > > Promises > > > > >>>>>> to > > > > >>>>>> finish the unfinished work are nice but not currency. Commits > > are > > > > >>>>>> currency. > > > > >>>>>> I hope at least the fault tolerance changes can be completed > and > > > > >>>>>> committed > > > > >>>>>> before we spin a 2.0 RC, and without causing a 2.0 release to > > slip > > > > >>>>>> further. > > > > >>
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
what happens now if the system has some kind of "partial > > > >>>>> completion state"? > > > >>>>> > > > >>>>> Documentation is one of those that is really hard to judge. We > want > > > to > > > >>>>> get > > > >>>>> this code out for people to use (and to free up our strained dev > > > >>>>> resources), but what good is some feature if the docs are > > > >>>>> missing/incomplete? > > > >>>>> > > > >>>>> I think I could stomach the docs being inaccurate (with a clear > > > >>>>> disclaimer > > > >>>>> that the chapter is incomplete -- that's a 5min task). But, I > > think I > > > >>>>> need > > > >>>>> an answer about how the feature handles our common dist-sys > > category > > > of > > > >>>>> problems before I can consider whether I'm ok with the feature > > > hitting > > > >>>>> 2.0... > > > >>>>> > > > >>>>> I'll also try to throw up a few nodes and play with it to address > > the > > > >>>>> problem as an (ignorant) user ;) > > > >>>>> > > > >>>>> > > > >>>>> Andrew Purtell wrote: > > > >>>>> > > > >>>>> I don't like that issues were identified as "blockers" but now > > there > > > is > > > >>>>> > > > >>>>>> an > > > >>>>>> attempt to walk that back. > > > >>>>>> > > > >>>>>> I don't like that development of this feature has lingered for a > > > long > > > >>>>>> time > > > >>>>>> in this unfinished state when this work could have been done by > > now, > > > >>>>>> now > > > >>>>>> that we are trying to get a 2.0 out the door. Because this is a > > > >>>>>> volunteer > > > >>>>>> project I cannot make any demand that it should be done, but I > can > > > >>>>>> certainly look at the current state and be nonplussed. This will > > be > > > >>>>>> yet > > > >>>>>> another half finished thing in 2.0 when this merge happens. > > Promises > > > >>>>>> to > > > >>>>>> finish the unfinished work are nice but not currency. Commits > are > > > >>>>>> currency. > > > >>>>>> I hope at least the fault tolerance changes can be completed and > > > >>>>>> committed > > > >>>>>> before we spin a 2.0 RC, and without causing a 2.0 release to > slip > > > >>>>>> further. > > > >>>>>> > > > >>>>>> Also, marking something experimental should be done on the > merits > > of > > > >>>>>> that > > > >>>>>> evaluation, not simply to justify dropping unfinished work into > a > > > >>>>>> release > > > >>>>>> branch. > > > >>>>>> > > > >>>>>> I will change my vote to -0. > > > >>>>>> > > > >>>>>> > > > >>>>>> On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar > > > >>>>>> wrote: > > > >>>>>> > > > >>>>>> I think there is some misconception of using the term "blockers" > > for > > > >>>>>> > > > >>>>>> referring to those jiras. My understanding is that those three > > jiras > > > >>>>>>> are > > > >>>>>>> blockers for the backup functionality to be more mature and > more > > > >>>>>>> usable. > > > >>>>>>> But they are not release blockers. Let's say we merged the code > > in, > > > >>>>>>> and > > > >>>>>>> for > > > >>>>>>> some reason those did not get addressed in ti
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
>>> problem as an (ignorant) user ;) > > >>>>> > > >>>>> > > >>>>> Andrew Purtell wrote: > > >>>>> > > >>>>> I don't like that issues were identified as "blockers" but now > there > > is > > >>>>> > > >>>>>> an > > >>>>>> attempt to walk that back. > > >>>>>> > > >>>>>> I don't like that development of this feature has lingered for a > > long > > >>>>>> time > > >>>>>> in this unfinished state when this work could have been done by > now, > > >>>>>> now > > >>>>>> that we are trying to get a 2.0 out the door. Because this is a > > >>>>>> volunteer > > >>>>>> project I cannot make any demand that it should be done, but I can > > >>>>>> certainly look at the current state and be nonplussed. This will > be > > >>>>>> yet > > >>>>>> another half finished thing in 2.0 when this merge happens. > Promises > > >>>>>> to > > >>>>>> finish the unfinished work are nice but not currency. Commits are > > >>>>>> currency. > > >>>>>> I hope at least the fault tolerance changes can be completed and > > >>>>>> committed > > >>>>>> before we spin a 2.0 RC, and without causing a 2.0 release to slip > > >>>>>> further. > > >>>>>> > > >>>>>> Also, marking something experimental should be done on the merits > of > > >>>>>> that > > >>>>>> evaluation, not simply to justify dropping unfinished work into a > > >>>>>> release > > >>>>>> branch. > > >>>>>> > > >>>>>> I will change my vote to -0. > > >>>>>> > > >>>>>> > > >>>>>> On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar > > >>>>>> wrote: > > >>>>>> > > >>>>>> I think there is some misconception of using the term "blockers" > for > > >>>>>> > > >>>>>> referring to those jiras. My understanding is that those three > jiras > > >>>>>>> are > > >>>>>>> blockers for the backup functionality to be more mature and more > > >>>>>>> usable. > > >>>>>>> But they are not release blockers. Let's say we merged the code > in, > > >>>>>>> and > > >>>>>>> for > > >>>>>>> some reason those did not get addressed in time. We can still do > > the > > >>>>>>> 2.0 > > >>>>>>> release without having to wait for the commits. We can instead > mark > > >>>>>>> the > > >>>>>>> "backup" feature as experimental with known issues and go on with > > the > > >>>>>>> release. In that sense they are not real release blockers. > > >>>>>>> > > >>>>>>> We are proposing the merge at this time because of the above that > > >>>>>>> maintaining this in a branch is becoming extremely costly and not > > >>>>>>> productive for the HBase community. Realistically, we cannot have > > the > > >>>>>>> luxury of having to wait another couple of months and doing yet > > >>>>>>> another > > >>>>>>> giant round of reviews because the code base is a moving target. > > >>>>>>> > > >>>>>>> Enis > > >>>>>>> > > >>>>>>> On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das< > d...@hortonworks.com> > > >>>>>>> wrote: > > >>>>>>> > > >>>>>>> Vlad, on the first point, I think what Stack is saying is that > > >>>>>>> creating > > >>>>>>> > > >>>>>>> the new branch (as Ted did) ignores the feedback incorporated > thus > > >>>>>>>> far > > >>>>>>>&
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
ency. Commits are > >>>>>> currency. > >>>>>> I hope at least the fault tolerance changes can be completed and > >>>>>> committed > >>>>>> before we spin a 2.0 RC, and without causing a 2.0 release to slip > >>>>>> further. > >>>>>> > >>>>>> Also, marking something experimental should be done on the merits of > >>>>>> that > >>>>>> evaluation, not simply to justify dropping unfinished work into a > >>>>>> release > >>>>>> branch. > >>>>>> > >>>>>> I will change my vote to -0. > >>>>>> > >>>>>> > >>>>>> On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar > >>>>>> wrote: > >>>>>> > >>>>>> I think there is some misconception of using the term "blockers" for > >>>>>> > >>>>>> referring to those jiras. My understanding is that those three jiras > >>>>>>> are > >>>>>>> blockers for the backup functionality to be more mature and more > >>>>>>> usable. > >>>>>>> But they are not release blockers. Let's say we merged the code in, > >>>>>>> and > >>>>>>> for > >>>>>>> some reason those did not get addressed in time. We can still do > the > >>>>>>> 2.0 > >>>>>>> release without having to wait for the commits. We can instead mark > >>>>>>> the > >>>>>>> "backup" feature as experimental with known issues and go on with > the > >>>>>>> release. In that sense they are not real release blockers. > >>>>>>> > >>>>>>> We are proposing the merge at this time because of the above that > >>>>>>> maintaining this in a branch is becoming extremely costly and not > >>>>>>> productive for the HBase community. Realistically, we cannot have > the > >>>>>>> luxury of having to wait another couple of months and doing yet > >>>>>>> another > >>>>>>> giant round of reviews because the code base is a moving target. > >>>>>>> > >>>>>>> Enis > >>>>>>> > >>>>>>> On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das > >>>>>>> wrote: > >>>>>>> > >>>>>>> Vlad, on the first point, I think what Stack is saying is that > >>>>>>> creating > >>>>>>> > >>>>>>> the new branch (as Ted did) ignores the feedback incorporated thus > >>>>>>>> far > >>>>>>>> in > >>>>>>>> the iterations of the mega-patch. That's a wrong way to go. > >>>>>>>> On the separation into a backup module, again, that was reverted > to > >>>>>>>> ease > >>>>>>>> reviews of the mega-patch, and was noted as work to be done > later. I > >>>>>>>> > >>>>>>>> think > >>>>>>>> > >>>>>>> Stack just wants to make the list of remaining work more complete > by > >>>>>>> > >>>>>>>> citing > >>>>>>>> > >>>>>>> that as pending work. > >>>>>>> > >>>>>>>> > >>>>>>>> From: Vladimir Rodionov > >>>>>>>> Sent: Monday, March 13, 2017 3:09 PM > >>>>>>>> To: dev@hbase.apache.org > >>>>>>>> Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote > >>>>>>>> closing > >>>>>>>> 3/11/2017 > >>>>>>>> > >>>>>>>> It ignores the feedback > >>>>>>>> > >>>>>>>> If I "ignore" feedback, I put my comment - why? I am always open > >>>>>>>>> for > >>>>>>>>> > >>>>>>>>> further discussions. If reviewer does not insist on
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
HBASE-15227 >>>>> currently >>>>> has a severity of BLOCKER. Despite what is going on in our politics, >>>>> words >>>>> matter and we do not get to redefine them for convenience. >>>>> >>>>> Once this work is merged in, how is HBASE-15227 not a blocker for 2.0? >>>>> Because Vlad offered to reduce its severity to make me feel better? >>>>> Currently the description on the issue is "System must be tolerant to >>>>> faults. Backup operations MUST be atomic (no partial completion state >>>>> in >>>>> system table)" Sounds like a blocker to me, indeed. It is an honest >>>>> assessment and I don't think anyone is doing the community a favor by >>>>> trying to walk that back. >>>>> >>>>> >>>>> On Tue, Mar 14, 2017 at 1:57 PM, Josh Elser >>>>> wrote: >>>>> >>>>> I took a moment to read through the "blockers" as originally >>>>> identified by >>>>> >>>>>> Vlad, and (to echo Enis' take) I read the majority of them as being >>>>>> blockers not for the next release, but for a "full-fledged feature". >>>>>> I'm >>>>>> going to intentional avoid addressing the discussion of shipping >>>>>> partial >>>>>> features (related, but not relevant at the moment). >>>>>> >>>>>> HBASE-15227 is actually the one that bothers me the most, with >>>>>> HBASE-17133 >>>>>> coming in close behind. Vlad, is there any documentation you can >>>>>> point me >>>>>> to about what the current issues are with the current implementation? >>>>>> For >>>>>> example, what happens now if the system has some kind of "partial >>>>>> completion state"? >>>>>> >>>>>> Documentation is one of those that is really hard to judge. We want to >>>>>> get >>>>>> this code out for people to use (and to free up our strained dev >>>>>> resources), but what good is some feature if the docs are >>>>>> missing/incomplete? >>>>>> >>>>>> I think I could stomach the docs being inaccurate (with a clear >>>>>> disclaimer >>>>>> that the chapter is incomplete -- that's a 5min task). But, I think I >>>>>> need >>>>>> an answer about how the feature handles our common dist-sys category >>>>>> of >>>>>> problems before I can consider whether I'm ok with the feature hitting >>>>>> 2.0... >>>>>> >>>>>> I'll also try to throw up a few nodes and play with it to address the >>>>>> problem as an (ignorant) user ;) >>>>>> >>>>>> >>>>>> Andrew Purtell wrote: >>>>>> >>>>>> I don't like that issues were identified as "blockers" but now there >>>>>> is >>>>>> >>>>>>> an >>>>>>> attempt to walk that back. >>>>>>> >>>>>>> I don't like that development of this feature has lingered for a long >>>>>>> time >>>>>>> in this unfinished state when this work could have been done by now, >>>>>>> now >>>>>>> that we are trying to get a 2.0 out the door. Because this is a >>>>>>> volunteer >>>>>>> project I cannot make any demand that it should be done, but I can >>>>>>> certainly look at the current state and be nonplussed. This will be >>>>>>> yet >>>>>>> another half finished thing in 2.0 when this merge happens. Promises >>>>>>> to >>>>>>> finish the unfinished work are nice but not currency. Commits are >>>>>>> currency. >>>>>>> I hope at least the fault tolerance changes can be completed and >>>>>>> committed >>>>>>> before we spin a 2.0 RC, and without causing a 2.0 release to slip >>>>>>> further. >>>>>>> >>>>>>> Also, marking something experimental should be done on the merits of >>>>>>> that >>>>>>
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
honest >>>> assessment and I don't think anyone is doing the community a favor by >>>> trying to walk that back. >>>> >>>> >>>> On Tue, Mar 14, 2017 at 1:57 PM, Josh Elser wrote: >>>> >>>> I took a moment to read through the "blockers" as originally identified >>>> by >>>> >>>>> Vlad, and (to echo Enis' take) I read the majority of them as being >>>>> blockers not for the next release, but for a "full-fledged feature". >>>>> I'm >>>>> going to intentional avoid addressing the discussion of shipping >>>>> partial >>>>> features (related, but not relevant at the moment). >>>>> >>>>> HBASE-15227 is actually the one that bothers me the most, with >>>>> HBASE-17133 >>>>> coming in close behind. Vlad, is there any documentation you can point >>>>> me >>>>> to about what the current issues are with the current implementation? >>>>> For >>>>> example, what happens now if the system has some kind of "partial >>>>> completion state"? >>>>> >>>>> Documentation is one of those that is really hard to judge. We want to >>>>> get >>>>> this code out for people to use (and to free up our strained dev >>>>> resources), but what good is some feature if the docs are >>>>> missing/incomplete? >>>>> >>>>> I think I could stomach the docs being inaccurate (with a clear >>>>> disclaimer >>>>> that the chapter is incomplete -- that's a 5min task). But, I think I >>>>> need >>>>> an answer about how the feature handles our common dist-sys category of >>>>> problems before I can consider whether I'm ok with the feature hitting >>>>> 2.0... >>>>> >>>>> I'll also try to throw up a few nodes and play with it to address the >>>>> problem as an (ignorant) user ;) >>>>> >>>>> >>>>> Andrew Purtell wrote: >>>>> >>>>> I don't like that issues were identified as "blockers" but now there is >>>>> >>>>>> an >>>>>> attempt to walk that back. >>>>>> >>>>>> I don't like that development of this feature has lingered for a long >>>>>> time >>>>>> in this unfinished state when this work could have been done by now, >>>>>> now >>>>>> that we are trying to get a 2.0 out the door. Because this is a >>>>>> volunteer >>>>>> project I cannot make any demand that it should be done, but I can >>>>>> certainly look at the current state and be nonplussed. This will be >>>>>> yet >>>>>> another half finished thing in 2.0 when this merge happens. Promises >>>>>> to >>>>>> finish the unfinished work are nice but not currency. Commits are >>>>>> currency. >>>>>> I hope at least the fault tolerance changes can be completed and >>>>>> committed >>>>>> before we spin a 2.0 RC, and without causing a 2.0 release to slip >>>>>> further. >>>>>> >>>>>> Also, marking something experimental should be done on the merits of >>>>>> that >>>>>> evaluation, not simply to justify dropping unfinished work into a >>>>>> release >>>>>> branch. >>>>>> >>>>>> I will change my vote to -0. >>>>>> >>>>>> >>>>>> On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar >>>>>> wrote: >>>>>> >>>>>> I think there is some misconception of using the term "blockers" for >>>>>> >>>>>> referring to those jiras. My understanding is that those three jiras >>>>>>> are >>>>>>> blockers for the backup functionality to be more mature and more >>>>>>> usable. >>>>>>> But they are not release blockers. Let's say we merged the code in, >>>>>>> and >>>>>>> for >>>>>>> some reason those did not get addressed in time. We can still do the >>>>>>> 2.0 >>
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
stomach the docs being inaccurate (with a clear disclaimer that the chapter is incomplete -- that's a 5min task). But, I think I need an answer about how the feature handles our common dist-sys category of problems before I can consider whether I'm ok with the feature hitting 2.0... I'll also try to throw up a few nodes and play with it to address the problem as an (ignorant) user ;) Andrew Purtell wrote: I don't like that issues were identified as "blockers" but now there is an attempt to walk that back. I don't like that development of this feature has lingered for a long time in this unfinished state when this work could have been done by now, now that we are trying to get a 2.0 out the door. Because this is a volunteer project I cannot make any demand that it should be done, but I can certainly look at the current state and be nonplussed. This will be yet another half finished thing in 2.0 when this merge happens. Promises to finish the unfinished work are nice but not currency. Commits are currency. I hope at least the fault tolerance changes can be completed and committed before we spin a 2.0 RC, and without causing a 2.0 release to slip further. Also, marking something experimental should be done on the merits of that evaluation, not simply to justify dropping unfinished work into a release branch. I will change my vote to -0. On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar wrote: I think there is some misconception of using the term "blockers" for referring to those jiras. My understanding is that those three jiras are blockers for the backup functionality to be more mature and more usable. But they are not release blockers. Let's say we merged the code in, and for some reason those did not get addressed in time. We can still do the 2.0 release without having to wait for the commits. We can instead mark the "backup" feature as experimental with known issues and go on with the release. In that sense they are not real release blockers. We are proposing the merge at this time because of the above that maintaining this in a branch is becoming extremely costly and not productive for the HBase community. Realistically, we cannot have the luxury of having to wait another couple of months and doing yet another giant round of reviews because the code base is a moving target. Enis On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das wrote: Vlad, on the first point, I think what Stack is saying is that creating the new branch (as Ted did) ignores the feedback incorporated thus far in the iterations of the mega-patch. That's a wrong way to go. On the separation into a backup module, again, that was reverted to ease reviews of the mega-patch, and was noted as work to be done later. I think Stack just wants to make the list of remaining work more complete by citing that as pending work. From: Vladimir Rodionov Sent: Monday, March 13, 2017 3:09 PM To: dev@hbase.apache.org Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017 It ignores the feedback If I "ignore" feedback, I put my comment - why? I am always open for further discussions. If reviewer does not insist on a particular request - it will be dropped. I think it is fair. he list is incomplete because a bunch of follow-ons came of the review cycle (including moving backup/restore out of core to live in its own module). For those who were not following our lengthy conversation on a review board, separation of a backup code into a separate module has been done last year, but has been reverted back by request of a reviewer. -Vladimir On Mon, Mar 13, 2017 at 2:23 PM, Stackwrote: On Fri, Mar 10, 2017 at 9:09 PM, Stackwrote: On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu wrote: HBASE-14123 branch has been created, with Vlad's mega patch v61. The patch put up for VOTE here was done on a branch. The call to merge seems to have been premature given the many cycles of review and test that happened subsequent (The cycles burned a bunch of dev resource). The patch as is is now in a state where it is too big for our infra; rb and JIRA are creaking under the size and # of iterations. Adding finish of new JIRAs to this merge implies a new round of review and test of an already massive patch. Who is going to do this work? Going back to a new branch seems wrong route to take. St.Ack To be more explicit, this patch was developed on a branch and then a bunch of dev resources were burned getting it into a state where it could be merged to master. Going back to a branch to bulk up the merge so it includes more JIRAs than the many it already incorporates is the wrong direction for us to be headed in. It ignores the feedback given and the work done by Vladimir slimming down an already over-broad scope. It is also predicated on abundant review and testing re
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
inished work are nice but not currency. Commits are >>>> currency. >>>> I hope at least the fault tolerance changes can be completed and >>>> committed >>>> before we spin a 2.0 RC, and without causing a 2.0 release to slip >>>> further. >>>> >>>> Also, marking something experimental should be done on the merits of >>>> that >>>> evaluation, not simply to justify dropping unfinished work into a >>>> release >>>> branch. >>>> >>>> I will change my vote to -0. >>>> >>>> >>>> On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar >>>> wrote: >>>> >>>> I think there is some misconception of using the term "blockers" for >>>> >>>>> referring to those jiras. My understanding is that those three jiras >>>>> are >>>>> blockers for the backup functionality to be more mature and more >>>>> usable. >>>>> But they are not release blockers. Let's say we merged the code in, and >>>>> for >>>>> some reason those did not get addressed in time. We can still do the >>>>> 2.0 >>>>> release without having to wait for the commits. We can instead mark the >>>>> "backup" feature as experimental with known issues and go on with the >>>>> release. In that sense they are not real release blockers. >>>>> >>>>> We are proposing the merge at this time because of the above that >>>>> maintaining this in a branch is becoming extremely costly and not >>>>> productive for the HBase community. Realistically, we cannot have the >>>>> luxury of having to wait another couple of months and doing yet another >>>>> giant round of reviews because the code base is a moving target. >>>>> >>>>> Enis >>>>> >>>>> On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das >>>>> wrote: >>>>> >>>>> Vlad, on the first point, I think what Stack is saying is that creating >>>>> >>>>>> the new branch (as Ted did) ignores the feedback incorporated thus far >>>>>> in >>>>>> the iterations of the mega-patch. That's a wrong way to go. >>>>>> On the separation into a backup module, again, that was reverted to >>>>>> ease >>>>>> reviews of the mega-patch, and was noted as work to be done later. I >>>>>> >>>>>> think >>>>> >>>>> Stack just wants to make the list of remaining work more complete by >>>>>> >>>>>> citing >>>>> >>>>> that as pending work. >>>>>> >>>>>> From: Vladimir Rodionov >>>>>> Sent: Monday, March 13, 2017 3:09 PM >>>>>> To: dev@hbase.apache.org >>>>>> Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing >>>>>> 3/11/2017 >>>>>> >>>>>>It ignores the feedback >>>>>> >>>>>>> If I "ignore" feedback, I put my comment - why? I am always open for >>>>>>> >>>>>> further discussions. If reviewer does not insist on a particular >>>>>> request >>>>>> >>>>>> - >>>>> >>>>> it will be dropped. I think it is fair. >>>>>> >>>>>> he list is incomplete because a bunch of >>>>>> >>>>>>> follow-ons came of the review cycle (including moving backup/restore >>>>>>>> >>>>>>>> out >>>>>>> >>>>>> of >>>>>> >>>>>> core to live in its own module). >>>>>>> For those who were not following our lengthy conversation on a review >>>>>>> >>>>>> board, separation of a backup code into a separate module has been >>>>>> done last year, but has been reverted back by request of a reviewer. >>>>>> >>>>>> >>>>>> -Vladimir >>>>>> >>>>>> On Mon, Mar 13, 2017 at 2:23 PM, Stack wrote: >>>>>> >>>>>> On Fri, Mar 10, 2017 at 9:09 PM, Stack wrote: >>>>>> >>&
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
g extremely costly and not productive for the HBase community. Realistically, we cannot have the luxury of having to wait another couple of months and doing yet another giant round of reviews because the code base is a moving target. Enis On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das wrote: Vlad, on the first point, I think what Stack is saying is that creating the new branch (as Ted did) ignores the feedback incorporated thus far in the iterations of the mega-patch. That's a wrong way to go. On the separation into a backup module, again, that was reverted to ease reviews of the mega-patch, and was noted as work to be done later. I think Stack just wants to make the list of remaining work more complete by citing that as pending work. ________________________ From: Vladimir Rodionov Sent: Monday, March 13, 2017 3:09 PM To: dev@hbase.apache.org Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017 It ignores the feedback If I "ignore" feedback, I put my comment - why? I am always open for further discussions. If reviewer does not insist on a particular request - it will be dropped. I think it is fair. he list is incomplete because a bunch of follow-ons came of the review cycle (including moving backup/restore out of core to live in its own module). For those who were not following our lengthy conversation on a review board, separation of a backup code into a separate module has been done last year, but has been reverted back by request of a reviewer. -Vladimir On Mon, Mar 13, 2017 at 2:23 PM, Stack wrote: On Fri, Mar 10, 2017 at 9:09 PM, Stack wrote: On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu wrote: HBASE-14123 branch has been created, with Vlad's mega patch v61. The patch put up for VOTE here was done on a branch. The call to merge seems to have been premature given the many cycles of review and test that happened subsequent (The cycles burned a bunch of dev resource). The patch as is is now in a state where it is too big for our infra; rb and JIRA are creaking under the size and # of iterations. Adding finish of new JIRAs to this merge implies a new round of review and test of an already massive patch. Who is going to do this work? Going back to a new branch seems wrong route to take. St.Ack To be more explicit, this patch was developed on a branch and then a bunch of dev resources were burned getting it into a state where it could be merged to master. Going back to a branch to bulk up the merge so it includes more JIRAs than the many it already incorporates is the wrong direction for us to be headed in. It ignores the feedback given and the work done by Vladimir slimming down an already over-broad scope. It is also predicated on abundant review and testing resource being on tap to cycle on a feature that is useful, but non-core. The patch is ready for merge IMO. Geoffrey makes a nice list of what is still to do though IIRC, the list is incomplete because a bunch of follow-ons came of the review cycle (including moving backup/restore out of core to live in its own module). The patch needs three votes to merge. I am not against merge but I am not voting for the patch because I do have any more time to spend on this non-core feature and feel that a vote will have me assume a responsibility I will not shirk. S FYI On Fri, Mar 10, 2017 at 3:30 PM, Ted Yu wrote: Thanks for the feedback, Andrew. How about the following plan: create branch HBASE-14123 off of master with mega patch v61 as the first commit (reviewed by Stack and Enis) Vlad and I continue development (the 3 blockers) based on HBASE-14123 branch when all of the blockers get +1 and merged into HBASE-14123 branch, we propose to community for merging into branch-2 (master branch, if branch-2 doesn't materialize for whatever reason) again Cheers On Fri, Mar 10, 2017 at 3:01 PM, Andrew Purtell< apurt...@apache.org> wrote: Thanks for the offer but I like that you were honest about compiling a list of issues that you thought were blockers for release. Since this proposal is a merge into 2.0, and we are trying to release 2.0, I am -1 on this merge until those blockers are addressed. I had a look at the list. I think the documentation issue is important but not actually a blocker. That may be a controversial opinion, but documentation can be back-filled worst case. So take HBASE-17133 off the list. Remaining are effectively HBASE-14417, HBASE-14141, and HBASE-15227. They all have patches attached to the respective JIRAs so completing this work won't be onerous. Get these committed and I will lift my -1. The others who voted +1 on this thread surely can help with that. Thanks. On Fri, Mar 10, 2017 at 2:32 PM, Vladimir Rodionov< vladrodio...@gmail.com> wrote: No problem I will downgrade Blockers to Majors if it scares you, Andrew [ima
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
jiras. My understanding is that those three jiras > are > >>> blockers for the backup functionality to be more mature and more > usable. > >>> But they are not release blockers. Let's say we merged the code in, and > >>> for > >>> some reason those did not get addressed in time. We can still do the > 2.0 > >>> release without having to wait for the commits. We can instead mark the > >>> "backup" feature as experimental with known issues and go on with the > >>> release. In that sense they are not real release blockers. > >>> > >>> We are proposing the merge at this time because of the above that > >>> maintaining this in a branch is becoming extremely costly and not > >>> productive for the HBase community. Realistically, we cannot have the > >>> luxury of having to wait another couple of months and doing yet another > >>> giant round of reviews because the code base is a moving target. > >>> > >>> Enis > >>> > >>> On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das > >>> wrote: > >>> > >>> Vlad, on the first point, I think what Stack is saying is that creating > >>>> the new branch (as Ted did) ignores the feedback incorporated thus far > >>>> in > >>>> the iterations of the mega-patch. That's a wrong way to go. > >>>> On the separation into a backup module, again, that was reverted to > ease > >>>> reviews of the mega-patch, and was noted as work to be done later. I > >>>> > >>> think > >>> > >>>> Stack just wants to make the list of remaining work more complete by > >>>> > >>> citing > >>> > >>>> that as pending work. > >>>> > >>>> From: Vladimir Rodionov > >>>> Sent: Monday, March 13, 2017 3:09 PM > >>>> To: dev@hbase.apache.org > >>>> Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing > >>>> 3/11/2017 > >>>> > >>>> It ignores the feedback > >>>>>> > >>>>> If I "ignore" feedback, I put my comment - why? I am always open for > >>>> further discussions. If reviewer does not insist on a particular > request > >>>> > >>> - > >>> > >>>> it will be dropped. I think it is fair. > >>>> > >>>> he list is incomplete because a bunch of > >>>>>> follow-ons came of the review cycle (including moving backup/restore > >>>>>> > >>>>> out > >>> > >>>> of > >>>> > >>>>> core to live in its own module). > >>>>>> > >>>>> For those who were not following our lengthy conversation on a review > >>>> board, separation of a backup code into a separate module has been > >>>> done last year, but has been reverted back by request of a reviewer. > >>>> > >>>> > >>>> -Vladimir > >>>> > >>>> On Mon, Mar 13, 2017 at 2:23 PM, Stack wrote: > >>>> > >>>> On Fri, Mar 10, 2017 at 9:09 PM, Stack wrote: > >>>>> > >>>>> On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu wrote: > >>>>>> > >>>>>> HBASE-14123 branch has been created, with Vlad's mega patch v61. > >>>>>>> > >>>>>>> The patch put up for VOTE here was done on a branch. The call to > >>>>>> > >>>>> merge > >>> > >>>> seems to have been premature given the many cycles of review and test > >>>>>> > >>>>> that > >>>>> > >>>>>> happened subsequent (The cycles burned a bunch of dev resource). > >>>>>> > >>>>>> The patch as is is now in a state where it is too big for our infra; > >>>>>> > >>>>> rb > >>> > >>>> and JIRA are creaking under the size and # of iterations. > >>>>>> > >>>>>> Adding finish of new JIRAs to this merge implies a new round of > >>>>>> > >>>>> review > >>> > >>>> and > >>>>> > >>>>>> t
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
ng yet another >>> giant round of reviews because the code base is a moving target. >>> >>> Enis >>> >>> On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das >>> wrote: >>> >>> Vlad, on the first point, I think what Stack is saying is that creating >>>> the new branch (as Ted did) ignores the feedback incorporated thus far >>>> in >>>> the iterations of the mega-patch. That's a wrong way to go. >>>> On the separation into a backup module, again, that was reverted to ease >>>> reviews of the mega-patch, and was noted as work to be done later. I >>>> >>> think >>> >>>> Stack just wants to make the list of remaining work more complete by >>>> >>> citing >>> >>>> that as pending work. >>>> >>>> From: Vladimir Rodionov >>>> Sent: Monday, March 13, 2017 3:09 PM >>>> To: dev@hbase.apache.org >>>> Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing >>>> 3/11/2017 >>>> >>>> It ignores the feedback >>>>>> >>>>> If I "ignore" feedback, I put my comment - why? I am always open for >>>> further discussions. If reviewer does not insist on a particular request >>>> >>> - >>> >>>> it will be dropped. I think it is fair. >>>> >>>> he list is incomplete because a bunch of >>>>>> follow-ons came of the review cycle (including moving backup/restore >>>>>> >>>>> out >>> >>>> of >>>> >>>>> core to live in its own module). >>>>>> >>>>> For those who were not following our lengthy conversation on a review >>>> board, separation of a backup code into a separate module has been >>>> done last year, but has been reverted back by request of a reviewer. >>>> >>>> >>>> -Vladimir >>>> >>>> On Mon, Mar 13, 2017 at 2:23 PM, Stack wrote: >>>> >>>> On Fri, Mar 10, 2017 at 9:09 PM, Stack wrote: >>>>> >>>>> On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu wrote: >>>>>> >>>>>> HBASE-14123 branch has been created, with Vlad's mega patch v61. >>>>>>> >>>>>>> The patch put up for VOTE here was done on a branch. The call to >>>>>> >>>>> merge >>> >>>> seems to have been premature given the many cycles of review and test >>>>>> >>>>> that >>>>> >>>>>> happened subsequent (The cycles burned a bunch of dev resource). >>>>>> >>>>>> The patch as is is now in a state where it is too big for our infra; >>>>>> >>>>> rb >>> >>>> and JIRA are creaking under the size and # of iterations. >>>>>> >>>>>> Adding finish of new JIRAs to this merge implies a new round of >>>>>> >>>>> review >>> >>>> and >>>>> >>>>>> test of an already massive patch. Who is going to do this work? >>>>>> >>>>>> Going back to a new branch seems wrong route to take. >>>>>> >>>>>> St.Ack >>>>>> >>>>>> >>>>>> >>>>>> To be more explicit, this patch was developed on a branch and then a >>>>> >>>> bunch >>>> >>>>> of dev resources were burned getting it into a state where it could be >>>>> merged to master. Going back to a branch to bulk up the merge so it >>>>> includes more JIRAs than the many it already incorporates is the wrong >>>>> direction for us to be headed in. It ignores the feedback given and the >>>>> work done by Vladimir slimming down an already over-broad scope. It is >>>>> >>>> also >>>> >>>>> predicated on abundant review and testing resource being on tap to >>>>> >>>> cycle >>> >>>> on >>>> >>>>> a feature that is useful, but non-core. >>>>> >>>>> The patch is ready for merge IMO. Geoffrey makes a nice list of what is >>>>> still to do though IIRC, the list is incomplete because a bunch
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
;>>> >>>> >>>> On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar >>>> wrote: >>>> >>>> I think there is some misconception of using the term "blockers" for >>>> >>>>> referring to those jiras. My understanding is that those three jiras >>>>> are >>>>> blockers for the backup functionality to be more mature and more >>>>> usable. >>>>> But they are not release blockers. Let's say we merged the code in, and >>>>> for >>>>> some reason those did not get addressed in time. We can still do the >>>>> 2.0 >>>>> release without having to wait for the commits. We can instead mark the >>>>> "backup" feature as experimental with known issues and go on with the >>>>> release. In that sense they are not real release blockers. >>>>> >>>>> We are proposing the merge at this time because of the above that >>>>> maintaining this in a branch is becoming extremely costly and not >>>>> productive for the HBase community. Realistically, we cannot have the >>>>> luxury of having to wait another couple of months and doing yet another >>>>> giant round of reviews because the code base is a moving target. >>>>> >>>>> Enis >>>>> >>>>> On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das >>>>> wrote: >>>>> >>>>> Vlad, on the first point, I think what Stack is saying is that creating >>>>> >>>>>> the new branch (as Ted did) ignores the feedback incorporated thus far >>>>>> in >>>>>> the iterations of the mega-patch. That's a wrong way to go. >>>>>> On the separation into a backup module, again, that was reverted to >>>>>> ease >>>>>> reviews of the mega-patch, and was noted as work to be done later. I >>>>>> >>>>>> think >>>>> >>>>> Stack just wants to make the list of remaining work more complete by >>>>>> >>>>>> citing >>>>> >>>>> that as pending work. >>>>>> >>>>>> From: Vladimir Rodionov >>>>>> Sent: Monday, March 13, 2017 3:09 PM >>>>>> To: dev@hbase.apache.org >>>>>> Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing >>>>>> 3/11/2017 >>>>>> >>>>>>It ignores the feedback >>>>>> >>>>>>> If I "ignore" feedback, I put my comment - why? I am always open for >>>>>>> >>>>>> further discussions. If reviewer does not insist on a particular >>>>>> request >>>>>> >>>>>> - >>>>> >>>>> it will be dropped. I think it is fair. >>>>>> >>>>>> he list is incomplete because a bunch of >>>>>> >>>>>>> follow-ons came of the review cycle (including moving backup/restore >>>>>>>> >>>>>>>> out >>>>>>> >>>>>> of >>>>>> >>>>>> core to live in its own module). >>>>>>> For those who were not following our lengthy conversation on a review >>>>>>> >>>>>> board, separation of a backup code into a separate module has been >>>>>> done last year, but has been reverted back by request of a reviewer. >>>>>> >>>>>> >>>>>> -Vladimir >>>>>> >>>>>> On Mon, Mar 13, 2017 at 2:23 PM, Stack wrote: >>>>>> >>>>>> On Fri, Mar 10, 2017 at 9:09 PM, Stack wrote: >>>>>> >>>>>>> On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu >>>>>>> wrote: >>>>>>> >>>>>>>> HBASE-14123 branch has been created, with Vlad's mega patch v61. >>>>>>>> >>>>>>>>> The patch put up for VOTE here was done on a branch. The call to >>>>>>>>> >>>>>>>> merge >>>>>>> >>>>>> seems to have been premature given the many cycles of review and test >>>>>> >>>>>>> that >&g
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
remaining work more complete by citing that as pending work. ________ From: Vladimir Rodionov Sent: Monday, March 13, 2017 3:09 PM To: dev@hbase.apache.org Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017 It ignores the feedback If I "ignore" feedback, I put my comment - why? I am always open for further discussions. If reviewer does not insist on a particular request - it will be dropped. I think it is fair. he list is incomplete because a bunch of follow-ons came of the review cycle (including moving backup/restore out of core to live in its own module). For those who were not following our lengthy conversation on a review board, separation of a backup code into a separate module has been done last year, but has been reverted back by request of a reviewer. -Vladimir On Mon, Mar 13, 2017 at 2:23 PM, Stack wrote: On Fri, Mar 10, 2017 at 9:09 PM, Stack wrote: On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu wrote: HBASE-14123 branch has been created, with Vlad's mega patch v61. The patch put up for VOTE here was done on a branch. The call to merge seems to have been premature given the many cycles of review and test that happened subsequent (The cycles burned a bunch of dev resource). The patch as is is now in a state where it is too big for our infra; rb and JIRA are creaking under the size and # of iterations. Adding finish of new JIRAs to this merge implies a new round of review and test of an already massive patch. Who is going to do this work? Going back to a new branch seems wrong route to take. St.Ack To be more explicit, this patch was developed on a branch and then a bunch of dev resources were burned getting it into a state where it could be merged to master. Going back to a branch to bulk up the merge so it includes more JIRAs than the many it already incorporates is the wrong direction for us to be headed in. It ignores the feedback given and the work done by Vladimir slimming down an already over-broad scope. It is also predicated on abundant review and testing resource being on tap to cycle on a feature that is useful, but non-core. The patch is ready for merge IMO. Geoffrey makes a nice list of what is still to do though IIRC, the list is incomplete because a bunch of follow-ons came of the review cycle (including moving backup/restore out of core to live in its own module). The patch needs three votes to merge. I am not against merge but I am not voting for the patch because I do have any more time to spend on this non-core feature and feel that a vote will have me assume a responsibility I will not shirk. S FYI On Fri, Mar 10, 2017 at 3:30 PM, Ted Yu wrote: Thanks for the feedback, Andrew. How about the following plan: create branch HBASE-14123 off of master with mega patch v61 as the first commit (reviewed by Stack and Enis) Vlad and I continue development (the 3 blockers) based on HBASE-14123 branch when all of the blockers get +1 and merged into HBASE-14123 branch, we propose to community for merging into branch-2 (master branch, if branch-2 doesn't materialize for whatever reason) again Cheers On Fri, Mar 10, 2017 at 3:01 PM, Andrew Purtell< apurt...@apache.org> wrote: Thanks for the offer but I like that you were honest about compiling a list of issues that you thought were blockers for release. Since this proposal is a merge into 2.0, and we are trying to release 2.0, I am -1 on this merge until those blockers are addressed. I had a look at the list. I think the documentation issue is important but not actually a blocker. That may be a controversial opinion, but documentation can be back-filled worst case. So take HBASE-17133 off the list. Remaining are effectively HBASE-14417, HBASE-14141, and HBASE-15227. They all have patches attached to the respective JIRAs so completing this work won't be onerous. Get these committed and I will lift my -1. The others who voted +1 on this thread surely can help with that. Thanks. On Fri, Mar 10, 2017 at 2:32 PM, Vladimir Rodionov< vladrodio...@gmail.com> wrote: No problem I will downgrade Blockers to Majors if it scares you, Andrew 🙂 Sent from my iPhone On Mar 10, 2017, at 1:52 PM, Andrew Purtell< apurt...@apache.org wrote: I know the merge of this feature has lagged substantially. I think that is regrettable but on another thread we are lamenting that 2.0 is already late. Unless I misunderstand, this is a proposal to merge something with known blockers into trunk before we branch it for 2.0 which will effectively prevent that release because these blockers will be there. I am inclined to veto. Probably we should not propose branch merges into code we are trying to get out the door with known blockers. Why not do that work first? It seems an obvious question. Perhaps I am missing som
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
did) ignores the feedback incorporated thus far >>>> in >>>> the iterations of the mega-patch. That's a wrong way to go. >>>> On the separation into a backup module, again, that was reverted to ease >>>> reviews of the mega-patch, and was noted as work to be done later. I >>>> >>> think >>> >>>> Stack just wants to make the list of remaining work more complete by >>>> >>> citing >>> >>>> that as pending work. >>>> >>>> From: Vladimir Rodionov >>>> Sent: Monday, March 13, 2017 3:09 PM >>>> To: dev@hbase.apache.org >>>> Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing >>>> 3/11/2017 >>>> >>>> It ignores the feedback >>>>>> >>>>> If I "ignore" feedback, I put my comment - why? I am always open for >>>> further discussions. If reviewer does not insist on a particular request >>>> >>> - >>> >>>> it will be dropped. I think it is fair. >>>> >>>> he list is incomplete because a bunch of >>>>>> follow-ons came of the review cycle (including moving backup/restore >>>>>> >>>>> out >>> >>>> of >>>> >>>>> core to live in its own module). >>>>>> >>>>> For those who were not following our lengthy conversation on a review >>>> board, separation of a backup code into a separate module has been >>>> done last year, but has been reverted back by request of a reviewer. >>>> >>>> >>>> -Vladimir >>>> >>>> On Mon, Mar 13, 2017 at 2:23 PM, Stack wrote: >>>> >>>> On Fri, Mar 10, 2017 at 9:09 PM, Stack wrote: >>>>> >>>>> On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu wrote: >>>>>> >>>>>> HBASE-14123 branch has been created, with Vlad's mega patch v61. >>>>>>> >>>>>>> The patch put up for VOTE here was done on a branch. The call to >>>>>> >>>>> merge >>> >>>> seems to have been premature given the many cycles of review and test >>>>>> >>>>> that >>>>> >>>>>> happened subsequent (The cycles burned a bunch of dev resource). >>>>>> >>>>>> The patch as is is now in a state where it is too big for our infra; >>>>>> >>>>> rb >>> >>>> and JIRA are creaking under the size and # of iterations. >>>>>> >>>>>> Adding finish of new JIRAs to this merge implies a new round of >>>>>> >>>>> review >>> >>>> and >>>>> >>>>>> test of an already massive patch. Who is going to do this work? >>>>>> >>>>>> Going back to a new branch seems wrong route to take. >>>>>> >>>>>> St.Ack >>>>>> >>>>>> >>>>>> >>>>>> To be more explicit, this patch was developed on a branch and then a >>>>> >>>> bunch >>>> >>>>> of dev resources were burned getting it into a state where it could be >>>>> merged to master. Going back to a branch to bulk up the merge so it >>>>> includes more JIRAs than the many it already incorporates is the wrong >>>>> direction for us to be headed in. It ignores the feedback given and the >>>>> work done by Vladimir slimming down an already over-broad scope. It is >>>>> >>>> also >>>> >>>>> predicated on abundant review and testing resource being on tap to >>>>> >>>> cycle >>> >>>> on >>>> >>>>> a feature that is useful, but non-core. >>>>> >>>>> The patch is ready for merge IMO. Geoffrey makes a nice list of what is >>>>> still to do though IIRC, the list is incomplete because a bunch of >>>>> follow-ons came of the review cycle (including moving backup/restore >>>>> >>>> out >>> >>>> of >>>> >>>>> core to live in its own module). >>>>> >>>>> The patch needs three votes to merge. I am not against merge but I am >>>>> &g
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
I took a moment to read through the "blockers" as originally identified by Vlad, and (to echo Enis' take) I read the majority of them as being blockers not for the next release, but for a "full-fledged feature". I'm going to intentional avoid addressing the discussion of shipping partial features (related, but not relevant at the moment). HBASE-15227 is actually the one that bothers me the most, with HBASE-17133 coming in close behind. Vlad, is there any documentation you can point me to about what the current issues are with the current implementation? For example, what happens now if the system has some kind of "partial completion state"? Documentation is one of those that is really hard to judge. We want to get this code out for people to use (and to free up our strained dev resources), but what good is some feature if the docs are missing/incomplete? I think I could stomach the docs being inaccurate (with a clear disclaimer that the chapter is incomplete -- that's a 5min task). But, I think I need an answer about how the feature handles our common dist-sys category of problems before I can consider whether I'm ok with the feature hitting 2.0... I'll also try to throw up a few nodes and play with it to address the problem as an (ignorant) user ;) Andrew Purtell wrote: I don't like that issues were identified as "blockers" but now there is an attempt to walk that back. I don't like that development of this feature has lingered for a long time in this unfinished state when this work could have been done by now, now that we are trying to get a 2.0 out the door. Because this is a volunteer project I cannot make any demand that it should be done, but I can certainly look at the current state and be nonplussed. This will be yet another half finished thing in 2.0 when this merge happens. Promises to finish the unfinished work are nice but not currency. Commits are currency. I hope at least the fault tolerance changes can be completed and committed before we spin a 2.0 RC, and without causing a 2.0 release to slip further. Also, marking something experimental should be done on the merits of that evaluation, not simply to justify dropping unfinished work into a release branch. I will change my vote to -0. On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar wrote: I think there is some misconception of using the term "blockers" for referring to those jiras. My understanding is that those three jiras are blockers for the backup functionality to be more mature and more usable. But they are not release blockers. Let's say we merged the code in, and for some reason those did not get addressed in time. We can still do the 2.0 release without having to wait for the commits. We can instead mark the "backup" feature as experimental with known issues and go on with the release. In that sense they are not real release blockers. We are proposing the merge at this time because of the above that maintaining this in a branch is becoming extremely costly and not productive for the HBase community. Realistically, we cannot have the luxury of having to wait another couple of months and doing yet another giant round of reviews because the code base is a moving target. Enis On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das wrote: Vlad, on the first point, I think what Stack is saying is that creating the new branch (as Ted did) ignores the feedback incorporated thus far in the iterations of the mega-patch. That's a wrong way to go. On the separation into a backup module, again, that was reverted to ease reviews of the mega-patch, and was noted as work to be done later. I think Stack just wants to make the list of remaining work more complete by citing that as pending work. From: Vladimir Rodionov Sent: Monday, March 13, 2017 3:09 PM To: dev@hbase.apache.org Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017 It ignores the feedback If I "ignore" feedback, I put my comment - why? I am always open for further discussions. If reviewer does not insist on a particular request - it will be dropped. I think it is fair. he list is incomplete because a bunch of follow-ons came of the review cycle (including moving backup/restore out of core to live in its own module). For those who were not following our lengthy conversation on a review board, separation of a backup code into a separate module has been done last year, but has been reverted back by request of a reviewer. -Vladimir On Mon, Mar 13, 2017 at 2:23 PM, Stack wrote: On Fri, Mar 10, 2017 at 9:09 PM, Stack wrote: On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu wrote: HBASE-14123 branch has been created, with Vlad's mega patch v61. The patch put up for VOTE here was done on a branch. The call to merge seems to have been premature given the
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
As of 6:06pm PST, I still saw these branches on https://github.com/apache/hbase/ Maybe there is some delay. Thanks Andrew. On Mon, Mar 13, 2017 at 5:59 PM, Andrew Purtell wrote: > That's because this was pushed with a different name. It's gone now. > > apurtell@onyx:~/src/hbase$ git branch -a | grep 14123 > remotes/origin/14123 > apurtell@onyx:~/src/hbase$ git push origin :14123 > Username for 'https://git-wip-us.apache.org': apurtell > Password for 'https://apurt...@git-wip-us.apache.org': > To https://git-wip-us.apache.org/repos/asf/hbase.git > - [deleted] 14123 > > > On Mon, Mar 13, 2017 at 4:13 PM, Ted Yu wrote: > > > I tried to delete the HBASE-14123 branch using commands I found on > > http://stackoverflow.com/questions/2003505/how-to- > > delete-a-git-branch-both-locally-and-remotely > > > > Not sure if there is lag on github side: > > > > $ git push origin :origin/HBASE-14123 > > error: unable to delete 'origin/HBASE-14123': remote ref does not exist > > error: failed to push some refs to ' > > https://git-wip-us.apache.org/repos/asf/hbase.git' > > > > FYI > > > > On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar wrote: > > > > > I think there is some misconception of using the term "blockers" for > > > referring to those jiras. My understanding is that those three jiras > are > > > blockers for the backup functionality to be more mature and more > usable. > > > But they are not release blockers. Let's say we merged the code in, and > > for > > > some reason those did not get addressed in time. We can still do the > 2.0 > > > release without having to wait for the commits. We can instead mark the > > > "backup" feature as experimental with known issues and go on with the > > > release. In that sense they are not real release blockers. > > > > > > We are proposing the merge at this time because of the above that > > > maintaining this in a branch is becoming extremely costly and not > > > productive for the HBase community. Realistically, we cannot have the > > > luxury of having to wait another couple of months and doing yet another > > > giant round of reviews because the code base is a moving target. > > > > > > Enis > > > > > > On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das > > wrote: > > > > > > > Vlad, on the first point, I think what Stack is saying is that > creating > > > > the new branch (as Ted did) ignores the feedback incorporated thus > far > > in > > > > the iterations of the mega-patch. That's a wrong way to go. > > > > On the separation into a backup module, again, that was reverted to > > ease > > > > reviews of the mega-patch, and was noted as work to be done later. I > > > think > > > > Stack just wants to make the list of remaining work more complete by > > > citing > > > > that as pending work. > > > > > > > > From: Vladimir Rodionov > > > > Sent: Monday, March 13, 2017 3:09 PM > > > > To: dev@hbase.apache.org > > > > Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote > closing > > > > 3/11/2017 > > > > > > > > >> It ignores the feedback > > > > > > > > If I "ignore" feedback, I put my comment - why? I am always open for > > > > further discussions. If reviewer does not insist on a particular > > request > > > - > > > > it will be dropped. I think it is fair. > > > > > > > > >> he list is incomplete because a bunch of > > > > >> follow-ons came of the review cycle (including moving > backup/restore > > > out > > > > of > > > > >> core to live in its own module). > > > > > > > > For those who were not following our lengthy conversation on a review > > > > board, separation of a backup code into a separate module has been > > > > done last year, but has been reverted back by request of a reviewer. > > > > > > > > > > > > -Vladimir > > > > > > > > On Mon, Mar 13, 2017 at 2:23 PM, Stack wrote: > > > > > > > > > On Fri, Mar 10, 2017 at 9:09 PM, Stack wrote: > > > > > > > > > > > On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu > > wrote: > > > > >
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
That's because this was pushed with a different name. It's gone now. apurtell@onyx:~/src/hbase$ git branch -a | grep 14123 remotes/origin/14123 apurtell@onyx:~/src/hbase$ git push origin :14123 Username for 'https://git-wip-us.apache.org': apurtell Password for 'https://apurt...@git-wip-us.apache.org': To https://git-wip-us.apache.org/repos/asf/hbase.git - [deleted] 14123 On Mon, Mar 13, 2017 at 4:13 PM, Ted Yu wrote: > I tried to delete the HBASE-14123 branch using commands I found on > http://stackoverflow.com/questions/2003505/how-to- > delete-a-git-branch-both-locally-and-remotely > > Not sure if there is lag on github side: > > $ git push origin :origin/HBASE-14123 > error: unable to delete 'origin/HBASE-14123': remote ref does not exist > error: failed to push some refs to ' > https://git-wip-us.apache.org/repos/asf/hbase.git' > > FYI > > On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar wrote: > > > I think there is some misconception of using the term "blockers" for > > referring to those jiras. My understanding is that those three jiras are > > blockers for the backup functionality to be more mature and more usable. > > But they are not release blockers. Let's say we merged the code in, and > for > > some reason those did not get addressed in time. We can still do the 2.0 > > release without having to wait for the commits. We can instead mark the > > "backup" feature as experimental with known issues and go on with the > > release. In that sense they are not real release blockers. > > > > We are proposing the merge at this time because of the above that > > maintaining this in a branch is becoming extremely costly and not > > productive for the HBase community. Realistically, we cannot have the > > luxury of having to wait another couple of months and doing yet another > > giant round of reviews because the code base is a moving target. > > > > Enis > > > > On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das > wrote: > > > > > Vlad, on the first point, I think what Stack is saying is that creating > > > the new branch (as Ted did) ignores the feedback incorporated thus far > in > > > the iterations of the mega-patch. That's a wrong way to go. > > > On the separation into a backup module, again, that was reverted to > ease > > > reviews of the mega-patch, and was noted as work to be done later. I > > think > > > Stack just wants to make the list of remaining work more complete by > > citing > > > that as pending work. > > > > > > From: Vladimir Rodionov > > > Sent: Monday, March 13, 2017 3:09 PM > > > To: dev@hbase.apache.org > > > Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing > > > 3/11/2017 > > > > > > >> It ignores the feedback > > > > > > If I "ignore" feedback, I put my comment - why? I am always open for > > > further discussions. If reviewer does not insist on a particular > request > > - > > > it will be dropped. I think it is fair. > > > > > > >> he list is incomplete because a bunch of > > > >> follow-ons came of the review cycle (including moving backup/restore > > out > > > of > > > >> core to live in its own module). > > > > > > For those who were not following our lengthy conversation on a review > > > board, separation of a backup code into a separate module has been > > > done last year, but has been reverted back by request of a reviewer. > > > > > > > > > -Vladimir > > > > > > On Mon, Mar 13, 2017 at 2:23 PM, Stack wrote: > > > > > > > On Fri, Mar 10, 2017 at 9:09 PM, Stack wrote: > > > > > > > > > On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu > wrote: > > > > > > > > > >> HBASE-14123 branch has been created, with Vlad's mega patch v61. > > > > >> > > > > > > > > > > The patch put up for VOTE here was done on a branch. The call to > > merge > > > > > seems to have been premature given the many cycles of review and > test > > > > that > > > > > happened subsequent (The cycles burned a bunch of dev resource). > > > > > > > > > > The patch as is is now in a state where it is too big for our > infra; > > rb > > > > > and JIRA are creaking under the size and # of iterations.
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
I don't like that issues were identified as "blockers" but now there is an attempt to walk that back. I don't like that development of this feature has lingered for a long time in this unfinished state when this work could have been done by now, now that we are trying to get a 2.0 out the door. Because this is a volunteer project I cannot make any demand that it should be done, but I can certainly look at the current state and be nonplussed. This will be yet another half finished thing in 2.0 when this merge happens. Promises to finish the unfinished work are nice but not currency. Commits are currency. I hope at least the fault tolerance changes can be completed and committed before we spin a 2.0 RC, and without causing a 2.0 release to slip further. Also, marking something experimental should be done on the merits of that evaluation, not simply to justify dropping unfinished work into a release branch. I will change my vote to -0. On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar wrote: > I think there is some misconception of using the term "blockers" for > referring to those jiras. My understanding is that those three jiras are > blockers for the backup functionality to be more mature and more usable. > But they are not release blockers. Let's say we merged the code in, and for > some reason those did not get addressed in time. We can still do the 2.0 > release without having to wait for the commits. We can instead mark the > "backup" feature as experimental with known issues and go on with the > release. In that sense they are not real release blockers. > > We are proposing the merge at this time because of the above that > maintaining this in a branch is becoming extremely costly and not > productive for the HBase community. Realistically, we cannot have the > luxury of having to wait another couple of months and doing yet another > giant round of reviews because the code base is a moving target. > > Enis > > On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das wrote: > > > Vlad, on the first point, I think what Stack is saying is that creating > > the new branch (as Ted did) ignores the feedback incorporated thus far in > > the iterations of the mega-patch. That's a wrong way to go. > > On the separation into a backup module, again, that was reverted to ease > > reviews of the mega-patch, and was noted as work to be done later. I > think > > Stack just wants to make the list of remaining work more complete by > citing > > that as pending work. > > ________________________ > > From: Vladimir Rodionov > > Sent: Monday, March 13, 2017 3:09 PM > > To: dev@hbase.apache.org > > Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing > > 3/11/2017 > > > > >> It ignores the feedback > > > > If I "ignore" feedback, I put my comment - why? I am always open for > > further discussions. If reviewer does not insist on a particular request > - > > it will be dropped. I think it is fair. > > > > >> he list is incomplete because a bunch of > > >> follow-ons came of the review cycle (including moving backup/restore > out > > of > > >> core to live in its own module). > > > > For those who were not following our lengthy conversation on a review > > board, separation of a backup code into a separate module has been > > done last year, but has been reverted back by request of a reviewer. > > > > > > -Vladimir > > > > On Mon, Mar 13, 2017 at 2:23 PM, Stack wrote: > > > > > On Fri, Mar 10, 2017 at 9:09 PM, Stack wrote: > > > > > > > On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu wrote: > > > > > > > >> HBASE-14123 branch has been created, with Vlad's mega patch v61. > > > >> > > > > > > > > The patch put up for VOTE here was done on a branch. The call to > merge > > > > seems to have been premature given the many cycles of review and test > > > that > > > > happened subsequent (The cycles burned a bunch of dev resource). > > > > > > > > The patch as is is now in a state where it is too big for our infra; > rb > > > > and JIRA are creaking under the size and # of iterations. > > > > > > > > Adding finish of new JIRAs to this merge implies a new round of > review > > > and > > > > test of an already massive patch. Who is going to do this work? > > > > > > > > Going back to a new branch seems wrong route to take. > > > > > > > > St.Ack > > > > > > > >
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
I tried to delete the HBASE-14123 branch using commands I found on http://stackoverflow.com/questions/2003505/how-to-delete-a-git-branch-both-locally-and-remotely Not sure if there is lag on github side: $ git push origin :origin/HBASE-14123 error: unable to delete 'origin/HBASE-14123': remote ref does not exist error: failed to push some refs to ' https://git-wip-us.apache.org/repos/asf/hbase.git' FYI On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar wrote: > I think there is some misconception of using the term "blockers" for > referring to those jiras. My understanding is that those three jiras are > blockers for the backup functionality to be more mature and more usable. > But they are not release blockers. Let's say we merged the code in, and for > some reason those did not get addressed in time. We can still do the 2.0 > release without having to wait for the commits. We can instead mark the > "backup" feature as experimental with known issues and go on with the > release. In that sense they are not real release blockers. > > We are proposing the merge at this time because of the above that > maintaining this in a branch is becoming extremely costly and not > productive for the HBase community. Realistically, we cannot have the > luxury of having to wait another couple of months and doing yet another > giant round of reviews because the code base is a moving target. > > Enis > > On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das wrote: > > > Vlad, on the first point, I think what Stack is saying is that creating > > the new branch (as Ted did) ignores the feedback incorporated thus far in > > the iterations of the mega-patch. That's a wrong way to go. > > On the separation into a backup module, again, that was reverted to ease > > reviews of the mega-patch, and was noted as work to be done later. I > think > > Stack just wants to make the list of remaining work more complete by > citing > > that as pending work. > > ________________________ > > From: Vladimir Rodionov > > Sent: Monday, March 13, 2017 3:09 PM > > To: dev@hbase.apache.org > > Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing > > 3/11/2017 > > > > >> It ignores the feedback > > > > If I "ignore" feedback, I put my comment - why? I am always open for > > further discussions. If reviewer does not insist on a particular request > - > > it will be dropped. I think it is fair. > > > > >> he list is incomplete because a bunch of > > >> follow-ons came of the review cycle (including moving backup/restore > out > > of > > >> core to live in its own module). > > > > For those who were not following our lengthy conversation on a review > > board, separation of a backup code into a separate module has been > > done last year, but has been reverted back by request of a reviewer. > > > > > > -Vladimir > > > > On Mon, Mar 13, 2017 at 2:23 PM, Stack wrote: > > > > > On Fri, Mar 10, 2017 at 9:09 PM, Stack wrote: > > > > > > > On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu wrote: > > > > > > > >> HBASE-14123 branch has been created, with Vlad's mega patch v61. > > > >> > > > > > > > > The patch put up for VOTE here was done on a branch. The call to > merge > > > > seems to have been premature given the many cycles of review and test > > > that > > > > happened subsequent (The cycles burned a bunch of dev resource). > > > > > > > > The patch as is is now in a state where it is too big for our infra; > rb > > > > and JIRA are creaking under the size and # of iterations. > > > > > > > > Adding finish of new JIRAs to this merge implies a new round of > review > > > and > > > > test of an already massive patch. Who is going to do this work? > > > > > > > > Going back to a new branch seems wrong route to take. > > > > > > > > St.Ack > > > > > > > > > > > > > > > To be more explicit, this patch was developed on a branch and then a > > bunch > > > of dev resources were burned getting it into a state where it could be > > > merged to master. Going back to a branch to bulk up the merge so it > > > includes more JIRAs than the many it already incorporates is the wrong > > > direction for us to be headed in. It ignores the feedback given and the > > > work done by Vladimir slimming down an already over-broad s
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
I think there is some misconception of using the term "blockers" for referring to those jiras. My understanding is that those three jiras are blockers for the backup functionality to be more mature and more usable. But they are not release blockers. Let's say we merged the code in, and for some reason those did not get addressed in time. We can still do the 2.0 release without having to wait for the commits. We can instead mark the "backup" feature as experimental with known issues and go on with the release. In that sense they are not real release blockers. We are proposing the merge at this time because of the above that maintaining this in a branch is becoming extremely costly and not productive for the HBase community. Realistically, we cannot have the luxury of having to wait another couple of months and doing yet another giant round of reviews because the code base is a moving target. Enis On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das wrote: > Vlad, on the first point, I think what Stack is saying is that creating > the new branch (as Ted did) ignores the feedback incorporated thus far in > the iterations of the mega-patch. That's a wrong way to go. > On the separation into a backup module, again, that was reverted to ease > reviews of the mega-patch, and was noted as work to be done later. I think > Stack just wants to make the list of remaining work more complete by citing > that as pending work. > > From: Vladimir Rodionov > Sent: Monday, March 13, 2017 3:09 PM > To: dev@hbase.apache.org > Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing > 3/11/2017 > > >> It ignores the feedback > > If I "ignore" feedback, I put my comment - why? I am always open for > further discussions. If reviewer does not insist on a particular request - > it will be dropped. I think it is fair. > > >> he list is incomplete because a bunch of > >> follow-ons came of the review cycle (including moving backup/restore out > of > >> core to live in its own module). > > For those who were not following our lengthy conversation on a review > board, separation of a backup code into a separate module has been > done last year, but has been reverted back by request of a reviewer. > > > -Vladimir > > On Mon, Mar 13, 2017 at 2:23 PM, Stack wrote: > > > On Fri, Mar 10, 2017 at 9:09 PM, Stack wrote: > > > > > On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu wrote: > > > > > >> HBASE-14123 branch has been created, with Vlad's mega patch v61. > > >> > > > > > > The patch put up for VOTE here was done on a branch. The call to merge > > > seems to have been premature given the many cycles of review and test > > that > > > happened subsequent (The cycles burned a bunch of dev resource). > > > > > > The patch as is is now in a state where it is too big for our infra; rb > > > and JIRA are creaking under the size and # of iterations. > > > > > > Adding finish of new JIRAs to this merge implies a new round of review > > and > > > test of an already massive patch. Who is going to do this work? > > > > > > Going back to a new branch seems wrong route to take. > > > > > > St.Ack > > > > > > > > > > > To be more explicit, this patch was developed on a branch and then a > bunch > > of dev resources were burned getting it into a state where it could be > > merged to master. Going back to a branch to bulk up the merge so it > > includes more JIRAs than the many it already incorporates is the wrong > > direction for us to be headed in. It ignores the feedback given and the > > work done by Vladimir slimming down an already over-broad scope. It is > also > > predicated on abundant review and testing resource being on tap to cycle > on > > a feature that is useful, but non-core. > > > > The patch is ready for merge IMO. Geoffrey makes a nice list of what is > > still to do though IIRC, the list is incomplete because a bunch of > > follow-ons came of the review cycle (including moving backup/restore out > of > > core to live in its own module). > > > > The patch needs three votes to merge. I am not against merge but I am not > > voting for the patch because I do have any more time to spend on this > > non-core feature and feel that a vote will have me assume a > responsibility > > I will not shirk. > > > > S > > > > > > > > > > > > > > > > > >> FYI > > >> > > >> On Fri, Mar 10, 2017 at 3:30 P
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
Vlad, on the first point, I think what Stack is saying is that creating the new branch (as Ted did) ignores the feedback incorporated thus far in the iterations of the mega-patch. That's a wrong way to go. On the separation into a backup module, again, that was reverted to ease reviews of the mega-patch, and was noted as work to be done later. I think Stack just wants to make the list of remaining work more complete by citing that as pending work. From: Vladimir Rodionov Sent: Monday, March 13, 2017 3:09 PM To: dev@hbase.apache.org Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017 >> It ignores the feedback If I "ignore" feedback, I put my comment - why? I am always open for further discussions. If reviewer does not insist on a particular request - it will be dropped. I think it is fair. >> he list is incomplete because a bunch of >> follow-ons came of the review cycle (including moving backup/restore out of >> core to live in its own module). For those who were not following our lengthy conversation on a review board, separation of a backup code into a separate module has been done last year, but has been reverted back by request of a reviewer. -Vladimir On Mon, Mar 13, 2017 at 2:23 PM, Stack wrote: > On Fri, Mar 10, 2017 at 9:09 PM, Stack wrote: > > > On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu wrote: > > > >> HBASE-14123 branch has been created, with Vlad's mega patch v61. > >> > > > > The patch put up for VOTE here was done on a branch. The call to merge > > seems to have been premature given the many cycles of review and test > that > > happened subsequent (The cycles burned a bunch of dev resource). > > > > The patch as is is now in a state where it is too big for our infra; rb > > and JIRA are creaking under the size and # of iterations. > > > > Adding finish of new JIRAs to this merge implies a new round of review > and > > test of an already massive patch. Who is going to do this work? > > > > Going back to a new branch seems wrong route to take. > > > > St.Ack > > > > > > > To be more explicit, this patch was developed on a branch and then a bunch > of dev resources were burned getting it into a state where it could be > merged to master. Going back to a branch to bulk up the merge so it > includes more JIRAs than the many it already incorporates is the wrong > direction for us to be headed in. It ignores the feedback given and the > work done by Vladimir slimming down an already over-broad scope. It is also > predicated on abundant review and testing resource being on tap to cycle on > a feature that is useful, but non-core. > > The patch is ready for merge IMO. Geoffrey makes a nice list of what is > still to do though IIRC, the list is incomplete because a bunch of > follow-ons came of the review cycle (including moving backup/restore out of > core to live in its own module). > > The patch needs three votes to merge. I am not against merge but I am not > voting for the patch because I do have any more time to spend on this > non-core feature and feel that a vote will have me assume a responsibility > I will not shirk. > > S > > > > > > > > > > >> FYI > >> > >> On Fri, Mar 10, 2017 at 3:30 PM, Ted Yu wrote: > >> > >> > Thanks for the feedback, Andrew. > >> > > >> > How about the following plan: > >> > > >> > create branch HBASE-14123 off of master with mega patch v61 as the > first > >> > commit (reviewed by Stack and Enis) > >> > Vlad and I continue development (the 3 blockers) based on HBASE-14123 > >> > branch > >> > when all of the blockers get +1 and merged into HBASE-14123 branch, we > >> > propose to community for merging into branch-2 (master branch, if > >> branch-2 > >> > doesn't materialize for whatever reason) again > >> > > >> > Cheers > >> > > >> > > >> > On Fri, Mar 10, 2017 at 3:01 PM, Andrew Purtell > >> > wrote: > >> > > >> >> Thanks for the offer but I like that you were honest about compiling > a > >> >> list > >> >> of issues that you thought were blockers for release. Since this > >> proposal > >> >> is a merge into 2.0, and we are trying to release 2.0, I am -1 on > this > >> >> merge until those blockers are addressed. > >> >> > >> >> I had a look at the list. > >> >> > >> >> I thin
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
>> It ignores the feedback If I "ignore" feedback, I put my comment - why? I am always open for further discussions. If reviewer does not insist on a particular request - it will be dropped. I think it is fair. >> he list is incomplete because a bunch of >> follow-ons came of the review cycle (including moving backup/restore out of >> core to live in its own module). For those who were not following our lengthy conversation on a review board, separation of a backup code into a separate module has been done last year, but has been reverted back by request of a reviewer. -Vladimir On Mon, Mar 13, 2017 at 2:23 PM, Stack wrote: > On Fri, Mar 10, 2017 at 9:09 PM, Stack wrote: > > > On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu wrote: > > > >> HBASE-14123 branch has been created, with Vlad's mega patch v61. > >> > > > > The patch put up for VOTE here was done on a branch. The call to merge > > seems to have been premature given the many cycles of review and test > that > > happened subsequent (The cycles burned a bunch of dev resource). > > > > The patch as is is now in a state where it is too big for our infra; rb > > and JIRA are creaking under the size and # of iterations. > > > > Adding finish of new JIRAs to this merge implies a new round of review > and > > test of an already massive patch. Who is going to do this work? > > > > Going back to a new branch seems wrong route to take. > > > > St.Ack > > > > > > > To be more explicit, this patch was developed on a branch and then a bunch > of dev resources were burned getting it into a state where it could be > merged to master. Going back to a branch to bulk up the merge so it > includes more JIRAs than the many it already incorporates is the wrong > direction for us to be headed in. It ignores the feedback given and the > work done by Vladimir slimming down an already over-broad scope. It is also > predicated on abundant review and testing resource being on tap to cycle on > a feature that is useful, but non-core. > > The patch is ready for merge IMO. Geoffrey makes a nice list of what is > still to do though IIRC, the list is incomplete because a bunch of > follow-ons came of the review cycle (including moving backup/restore out of > core to live in its own module). > > The patch needs three votes to merge. I am not against merge but I am not > voting for the patch because I do have any more time to spend on this > non-core feature and feel that a vote will have me assume a responsibility > I will not shirk. > > S > > > > > > > > > > >> FYI > >> > >> On Fri, Mar 10, 2017 at 3:30 PM, Ted Yu wrote: > >> > >> > Thanks for the feedback, Andrew. > >> > > >> > How about the following plan: > >> > > >> > create branch HBASE-14123 off of master with mega patch v61 as the > first > >> > commit (reviewed by Stack and Enis) > >> > Vlad and I continue development (the 3 blockers) based on HBASE-14123 > >> > branch > >> > when all of the blockers get +1 and merged into HBASE-14123 branch, we > >> > propose to community for merging into branch-2 (master branch, if > >> branch-2 > >> > doesn't materialize for whatever reason) again > >> > > >> > Cheers > >> > > >> > > >> > On Fri, Mar 10, 2017 at 3:01 PM, Andrew Purtell > >> > wrote: > >> > > >> >> Thanks for the offer but I like that you were honest about compiling > a > >> >> list > >> >> of issues that you thought were blockers for release. Since this > >> proposal > >> >> is a merge into 2.0, and we are trying to release 2.0, I am -1 on > this > >> >> merge until those blockers are addressed. > >> >> > >> >> I had a look at the list. > >> >> > >> >> I think the documentation issue is important but not actually a > >> blocker. > >> >> That may be a controversial opinion, but documentation can be > >> back-filled > >> >> worst case. So take HBASE-17133 off the list. > >> >> > >> >> Remaining are effectively HBASE-14417, HBASE-14141, and HBASE-15227. > >> They > >> >> all have patches attached to the respective JIRAs so completing this > >> work > >> >> won't be onerous. Get these committed and I will lift my -1. The > others > >> >> who > >> >> voted +1 on this thread surely can help with that. > >> >> > >> >> Thanks. > >> >> > >> >> > >> >> On Fri, Mar 10, 2017 at 2:32 PM, Vladimir Rodionov < > >> >> vladrodio...@gmail.com> > >> >> wrote: > >> >> > >> >> > No problem I will downgrade Blockers to Majors if it scares you, > >> Andrew > >> >> 🙂 > >> >> > > >> >> > Sent from my iPhone > >> >> > > >> >> > > On Mar 10, 2017, at 1:52 PM, Andrew Purtell > > >> >> wrote: > >> >> > > > >> >> > > I know the merge of this feature has lagged substantially. I > think > >> >> that > >> >> > is > >> >> > > regrettable but on another thread we are lamenting that 2.0 is > >> already > >> >> > > late. Unless I misunderstand, this is a proposal to merge > something > >> >> with > >> >> > > known blockers into trunk before we branch it for 2.0 which will > >> >> > > effectively prevent that release because these blockers will be
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
On Fri, Mar 10, 2017 at 9:09 PM, Stack wrote: > On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu wrote: > >> HBASE-14123 branch has been created, with Vlad's mega patch v61. >> > > The patch put up for VOTE here was done on a branch. The call to merge > seems to have been premature given the many cycles of review and test that > happened subsequent (The cycles burned a bunch of dev resource). > > The patch as is is now in a state where it is too big for our infra; rb > and JIRA are creaking under the size and # of iterations. > > Adding finish of new JIRAs to this merge implies a new round of review and > test of an already massive patch. Who is going to do this work? > > Going back to a new branch seems wrong route to take. > > St.Ack > > > To be more explicit, this patch was developed on a branch and then a bunch of dev resources were burned getting it into a state where it could be merged to master. Going back to a branch to bulk up the merge so it includes more JIRAs than the many it already incorporates is the wrong direction for us to be headed in. It ignores the feedback given and the work done by Vladimir slimming down an already over-broad scope. It is also predicated on abundant review and testing resource being on tap to cycle on a feature that is useful, but non-core. The patch is ready for merge IMO. Geoffrey makes a nice list of what is still to do though IIRC, the list is incomplete because a bunch of follow-ons came of the review cycle (including moving backup/restore out of core to live in its own module). The patch needs three votes to merge. I am not against merge but I am not voting for the patch because I do have any more time to spend on this non-core feature and feel that a vote will have me assume a responsibility I will not shirk. S > > > >> FYI >> >> On Fri, Mar 10, 2017 at 3:30 PM, Ted Yu wrote: >> >> > Thanks for the feedback, Andrew. >> > >> > How about the following plan: >> > >> > create branch HBASE-14123 off of master with mega patch v61 as the first >> > commit (reviewed by Stack and Enis) >> > Vlad and I continue development (the 3 blockers) based on HBASE-14123 >> > branch >> > when all of the blockers get +1 and merged into HBASE-14123 branch, we >> > propose to community for merging into branch-2 (master branch, if >> branch-2 >> > doesn't materialize for whatever reason) again >> > >> > Cheers >> > >> > >> > On Fri, Mar 10, 2017 at 3:01 PM, Andrew Purtell >> > wrote: >> > >> >> Thanks for the offer but I like that you were honest about compiling a >> >> list >> >> of issues that you thought were blockers for release. Since this >> proposal >> >> is a merge into 2.0, and we are trying to release 2.0, I am -1 on this >> >> merge until those blockers are addressed. >> >> >> >> I had a look at the list. >> >> >> >> I think the documentation issue is important but not actually a >> blocker. >> >> That may be a controversial opinion, but documentation can be >> back-filled >> >> worst case. So take HBASE-17133 off the list. >> >> >> >> Remaining are effectively HBASE-14417, HBASE-14141, and HBASE-15227. >> They >> >> all have patches attached to the respective JIRAs so completing this >> work >> >> won't be onerous. Get these committed and I will lift my -1. The others >> >> who >> >> voted +1 on this thread surely can help with that. >> >> >> >> Thanks. >> >> >> >> >> >> On Fri, Mar 10, 2017 at 2:32 PM, Vladimir Rodionov < >> >> vladrodio...@gmail.com> >> >> wrote: >> >> >> >> > No problem I will downgrade Blockers to Majors if it scares you, >> Andrew >> >> 🙂 >> >> > >> >> > Sent from my iPhone >> >> > >> >> > > On Mar 10, 2017, at 1:52 PM, Andrew Purtell >> >> wrote: >> >> > > >> >> > > I know the merge of this feature has lagged substantially. I think >> >> that >> >> > is >> >> > > regrettable but on another thread we are lamenting that 2.0 is >> already >> >> > > late. Unless I misunderstand, this is a proposal to merge something >> >> with >> >> > > known blockers into trunk before we branch it for 2.0 which will >> >> > > effectively prevent that release because these blockers will be >> >> there. I >> >> > am >> >> > > inclined to veto. Probably we should not propose branch merges into >> >> code >> >> > we >> >> > > are trying to get out the door with known blockers. Why not do that >> >> work >> >> > > first? It seems an obvious question. Perhaps I am missing >> something. >> >> > > >> >> > > If we can branch for 2.0 now and then merge this, and not into the >> 2.0 >> >> > > branch, I would vote +1 for branch merge even with known blockers >> >> > pending. >> >> > > >> >> > > >> >> > > On Fri, Mar 10, 2017 at 1:42 PM, Vladimir Rodionov < >> >> > vladrodio...@gmail.com> >> >> > > wrote: >> >> > > >> >> > >> They are not blockers for merge - only for 2.0. GA >> >> > >> As I said already the feature is usable right now >> >> > >> We would like to continue working on master and we would like to >> see >> >> a >> >> > >> commitment from community >> >> > >> >>
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu wrote: > HBASE-14123 branch has been created, with Vlad's mega patch v61. > The patch put up for VOTE here was done on a branch. The call to merge seems to have been premature given the many cycles of review and test that happened subsequent (The cycles burned a bunch of dev resource). The patch as is is now in a state where it is too big for our infra; rb and JIRA are creaking under the size and # of iterations. Adding finish of new JIRAs to this merge implies a new round of review and test of an already massive patch. Who is going to do this work? Going back to a new branch seems wrong route to take. St.Ack > FYI > > On Fri, Mar 10, 2017 at 3:30 PM, Ted Yu wrote: > > > Thanks for the feedback, Andrew. > > > > How about the following plan: > > > > create branch HBASE-14123 off of master with mega patch v61 as the first > > commit (reviewed by Stack and Enis) > > Vlad and I continue development (the 3 blockers) based on HBASE-14123 > > branch > > when all of the blockers get +1 and merged into HBASE-14123 branch, we > > propose to community for merging into branch-2 (master branch, if > branch-2 > > doesn't materialize for whatever reason) again > > > > Cheers > > > > > > On Fri, Mar 10, 2017 at 3:01 PM, Andrew Purtell > > wrote: > > > >> Thanks for the offer but I like that you were honest about compiling a > >> list > >> of issues that you thought were blockers for release. Since this > proposal > >> is a merge into 2.0, and we are trying to release 2.0, I am -1 on this > >> merge until those blockers are addressed. > >> > >> I had a look at the list. > >> > >> I think the documentation issue is important but not actually a blocker. > >> That may be a controversial opinion, but documentation can be > back-filled > >> worst case. So take HBASE-17133 off the list. > >> > >> Remaining are effectively HBASE-14417, HBASE-14141, and HBASE-15227. > They > >> all have patches attached to the respective JIRAs so completing this > work > >> won't be onerous. Get these committed and I will lift my -1. The others > >> who > >> voted +1 on this thread surely can help with that. > >> > >> Thanks. > >> > >> > >> On Fri, Mar 10, 2017 at 2:32 PM, Vladimir Rodionov < > >> vladrodio...@gmail.com> > >> wrote: > >> > >> > No problem I will downgrade Blockers to Majors if it scares you, > Andrew > >> 🙂 > >> > > >> > Sent from my iPhone > >> > > >> > > On Mar 10, 2017, at 1:52 PM, Andrew Purtell > >> wrote: > >> > > > >> > > I know the merge of this feature has lagged substantially. I think > >> that > >> > is > >> > > regrettable but on another thread we are lamenting that 2.0 is > already > >> > > late. Unless I misunderstand, this is a proposal to merge something > >> with > >> > > known blockers into trunk before we branch it for 2.0 which will > >> > > effectively prevent that release because these blockers will be > >> there. I > >> > am > >> > > inclined to veto. Probably we should not propose branch merges into > >> code > >> > we > >> > > are trying to get out the door with known blockers. Why not do that > >> work > >> > > first? It seems an obvious question. Perhaps I am missing something. > >> > > > >> > > If we can branch for 2.0 now and then merge this, and not into the > 2.0 > >> > > branch, I would vote +1 for branch merge even with known blockers > >> > pending. > >> > > > >> > > > >> > > On Fri, Mar 10, 2017 at 1:42 PM, Vladimir Rodionov < > >> > vladrodio...@gmail.com> > >> > > wrote: > >> > > > >> > >> They are not blockers for merge - only for 2.0. GA > >> > >> As I said already the feature is usable right now > >> > >> We would like to continue working on master and we would like to > see > >> a > >> > >> commitment from community > >> > >> > >> > >> Sent from my iPhone > >> > >> > >> > >> On Mar 10, 2017, at 11:16 AM, Andrew Purtell > >> > wrote: > >> > >> > >> > Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release. > >> > >>> > >> > >>> If we have identified blockers, why merge this before they are in? > >> > >>> Otherwise we can't release 2.0, and it is overdue. > >> > >>> > >> > >>> > >> > >>> On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov < > >> > >> vladrodio...@gmail.com> > >> > >>> wrote: > >> > >>> > >> > Hello, HBase folks > >> > > >> > For your consideration today is Backup/Restore feature for Apache > >> > HBAse > >> > 2.0. > >> > Backup code is available as a mega patch in HBASE-14123 (v61), > >> applies > >> > cleanly to the current master, all test PASS, patch has no other > >> > issues. > >> > > >> > The patch has gone through numerous rounds of code reviews and > has > >> > >> probably > >> > the most lengthy discussion thread on Apache JIRA (HBASE-14123) > :) > >> > > >> > The work has been split into 3 phases (HBASE-14030, 14123, 14414) > >> Two > >> > >> first > >> > are complete, third one is still in progress. > >> > > >> > > >> >
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
HBASE-14123 branch has been created, with Vlad's mega patch v61. FYI On Fri, Mar 10, 2017 at 3:30 PM, Ted Yu wrote: > Thanks for the feedback, Andrew. > > How about the following plan: > > create branch HBASE-14123 off of master with mega patch v61 as the first > commit (reviewed by Stack and Enis) > Vlad and I continue development (the 3 blockers) based on HBASE-14123 > branch > when all of the blockers get +1 and merged into HBASE-14123 branch, we > propose to community for merging into branch-2 (master branch, if branch-2 > doesn't materialize for whatever reason) again > > Cheers > > > On Fri, Mar 10, 2017 at 3:01 PM, Andrew Purtell > wrote: > >> Thanks for the offer but I like that you were honest about compiling a >> list >> of issues that you thought were blockers for release. Since this proposal >> is a merge into 2.0, and we are trying to release 2.0, I am -1 on this >> merge until those blockers are addressed. >> >> I had a look at the list. >> >> I think the documentation issue is important but not actually a blocker. >> That may be a controversial opinion, but documentation can be back-filled >> worst case. So take HBASE-17133 off the list. >> >> Remaining are effectively HBASE-14417, HBASE-14141, and HBASE-15227. They >> all have patches attached to the respective JIRAs so completing this work >> won't be onerous. Get these committed and I will lift my -1. The others >> who >> voted +1 on this thread surely can help with that. >> >> Thanks. >> >> >> On Fri, Mar 10, 2017 at 2:32 PM, Vladimir Rodionov < >> vladrodio...@gmail.com> >> wrote: >> >> > No problem I will downgrade Blockers to Majors if it scares you, Andrew >> 🙂 >> > >> > Sent from my iPhone >> > >> > > On Mar 10, 2017, at 1:52 PM, Andrew Purtell >> wrote: >> > > >> > > I know the merge of this feature has lagged substantially. I think >> that >> > is >> > > regrettable but on another thread we are lamenting that 2.0 is already >> > > late. Unless I misunderstand, this is a proposal to merge something >> with >> > > known blockers into trunk before we branch it for 2.0 which will >> > > effectively prevent that release because these blockers will be >> there. I >> > am >> > > inclined to veto. Probably we should not propose branch merges into >> code >> > we >> > > are trying to get out the door with known blockers. Why not do that >> work >> > > first? It seems an obvious question. Perhaps I am missing something. >> > > >> > > If we can branch for 2.0 now and then merge this, and not into the 2.0 >> > > branch, I would vote +1 for branch merge even with known blockers >> > pending. >> > > >> > > >> > > On Fri, Mar 10, 2017 at 1:42 PM, Vladimir Rodionov < >> > vladrodio...@gmail.com> >> > > wrote: >> > > >> > >> They are not blockers for merge - only for 2.0. GA >> > >> As I said already the feature is usable right now >> > >> We would like to continue working on master and we would like to see >> a >> > >> commitment from community >> > >> >> > >> Sent from my iPhone >> > >> >> > >> On Mar 10, 2017, at 11:16 AM, Andrew Purtell >> > wrote: >> > >> >> > Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release. >> > >>> >> > >>> If we have identified blockers, why merge this before they are in? >> > >>> Otherwise we can't release 2.0, and it is overdue. >> > >>> >> > >>> >> > >>> On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov < >> > >> vladrodio...@gmail.com> >> > >>> wrote: >> > >>> >> > Hello, HBase folks >> > >> > For your consideration today is Backup/Restore feature for Apache >> > HBAse >> > 2.0. >> > Backup code is available as a mega patch in HBASE-14123 (v61), >> applies >> > cleanly to the current master, all test PASS, patch has no other >> > issues. >> > >> > The patch has gone through numerous rounds of code reviews and has >> > >> probably >> > the most lengthy discussion thread on Apache JIRA (HBASE-14123) :) >> > >> > The work has been split into 3 phases (HBASE-14030, 14123, 14414) >> Two >> > >> first >> > are complete, third one is still in progress. >> > >> > >> > *** Summary of work HBASE-14123 >> > >> > The new feature introduces new command-line extensions to the hbase >> > >> command >> > and, from the client side, is accessible through command-line only >> > Operations: >> > * Create full backup on a list of tables or backup set >> > * Create incremental backup image for table list or backup set >> > * Restore list of tables from a given backup image >> > * Show current backup progress >> > * Delete backup image and all related images >> > * Show history of backups >> > * Backup set operations: create backup set, add/remove table >> to/from >> > >> backup >> > set, etc >> > >> > In the current implementation, the feature is already usable, >> meaning >> > >> that >> > users can backup tables and restore them using provided >> command-line
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
Thanks for the feedback, Andrew. How about the following plan: create branch HBASE-14123 off of master with mega patch v61 as the first commit (reviewed by Stack and Enis) Vlad and I continue development (the 3 blockers) based on HBASE-14123 branch when all of the blockers get +1 and merged into HBASE-14123 branch, we propose to community for merging into branch-2 (master branch, if branch-2 doesn't materialize for whatever reason) again Cheers On Fri, Mar 10, 2017 at 3:01 PM, Andrew Purtell wrote: > Thanks for the offer but I like that you were honest about compiling a list > of issues that you thought were blockers for release. Since this proposal > is a merge into 2.0, and we are trying to release 2.0, I am -1 on this > merge until those blockers are addressed. > > I had a look at the list. > > I think the documentation issue is important but not actually a blocker. > That may be a controversial opinion, but documentation can be back-filled > worst case. So take HBASE-17133 off the list. > > Remaining are effectively HBASE-14417, HBASE-14141, and HBASE-15227. They > all have patches attached to the respective JIRAs so completing this work > won't be onerous. Get these committed and I will lift my -1. The others who > voted +1 on this thread surely can help with that. > > Thanks. > > > On Fri, Mar 10, 2017 at 2:32 PM, Vladimir Rodionov > > wrote: > > > No problem I will downgrade Blockers to Majors if it scares you, Andrew > 🙂 > > > > Sent from my iPhone > > > > > On Mar 10, 2017, at 1:52 PM, Andrew Purtell > wrote: > > > > > > I know the merge of this feature has lagged substantially. I think > that > > is > > > regrettable but on another thread we are lamenting that 2.0 is already > > > late. Unless I misunderstand, this is a proposal to merge something > with > > > known blockers into trunk before we branch it for 2.0 which will > > > effectively prevent that release because these blockers will be there. > I > > am > > > inclined to veto. Probably we should not propose branch merges into > code > > we > > > are trying to get out the door with known blockers. Why not do that > work > > > first? It seems an obvious question. Perhaps I am missing something. > > > > > > If we can branch for 2.0 now and then merge this, and not into the 2.0 > > > branch, I would vote +1 for branch merge even with known blockers > > pending. > > > > > > > > > On Fri, Mar 10, 2017 at 1:42 PM, Vladimir Rodionov < > > vladrodio...@gmail.com> > > > wrote: > > > > > >> They are not blockers for merge - only for 2.0. GA > > >> As I said already the feature is usable right now > > >> We would like to continue working on master and we would like to see a > > >> commitment from community > > >> > > >> Sent from my iPhone > > >> > > >> On Mar 10, 2017, at 11:16 AM, Andrew Purtell > > wrote: > > >> > > Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release. > > >>> > > >>> If we have identified blockers, why merge this before they are in? > > >>> Otherwise we can't release 2.0, and it is overdue. > > >>> > > >>> > > >>> On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov < > > >> vladrodio...@gmail.com> > > >>> wrote: > > >>> > > Hello, HBase folks > > > > For your consideration today is Backup/Restore feature for Apache > > HBAse > > 2.0. > > Backup code is available as a mega patch in HBASE-14123 (v61), > applies > > cleanly to the current master, all test PASS, patch has no other > > issues. > > > > The patch has gone through numerous rounds of code reviews and has > > >> probably > > the most lengthy discussion thread on Apache JIRA (HBASE-14123) :) > > > > The work has been split into 3 phases (HBASE-14030, 14123, 14414) > Two > > >> first > > are complete, third one is still in progress. > > > > > > *** Summary of work HBASE-14123 > > > > The new feature introduces new command-line extensions to the hbase > > >> command > > and, from the client side, is accessible through command-line only > > Operations: > > * Create full backup on a list of tables or backup set > > * Create incremental backup image for table list or backup set > > * Restore list of tables from a given backup image > > * Show current backup progress > > * Delete backup image and all related images > > * Show history of backups > > * Backup set operations: create backup set, add/remove table to/from > > >> backup > > set, etc > > > > In the current implementation, the feature is already usable, > meaning > > >> that > > users can backup tables and restore them using provided command-line > > >> tools. > > Both: full and incremental backups are supported. > > This work is based on original work of IBM team (HBASE-7912). The > full > > >> list > > of JIRAs included in this mega patch can be found in three umbrella > > >> JIRAs: > > HBASE-14030 (Phase 1), HBASE-14123 (Phas
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
I have no vote here, but I'd argue that HBASE-14417 and HBASE-14141 shouldn't be blockers. I agree that HBASE-15227 to add fault tolerance is a blocker. HBASE-14417 is support for incrementally backing up bulk loaded rows. That's an important feature, but if you don't use bulk loads, or don't care about _incremental_ backups of bulk loads, you'd be able to use backup quite happily without it. Even if you bulk load occasionally, you can do a full backup afterward. In the meantime, the docs and help text would just need to make the limitation clear in big bold letters. :-) HBASE-14141 is allowing HBase Backup to filter out unnecessary data from its incremental backups. Since the backup tool allows you to specify that only certain tables be backed up, incremental backups at WAL granularity will accidentally backup some rows from unneeded tables. This doesn't affect the correctness of the to-be-backed-up tables backups or restores, only its storage cost. The feature works, but there's an important storage optimization to be done. As someone who works on clusters that don't use bulk load, and would want to backup all tables, neither of these seems like a showstopper. However, if HBASE-14141 would be a breaking change to the backup format, then that would change my mind about it being a blocker. Geoffrey On Fri, Mar 10, 2017 at 3:01 PM, Andrew Purtell wrote: > Thanks for the offer but I like that you were honest about compiling a list > of issues that you thought were blockers for release. Since this proposal > is a merge into 2.0, and we are trying to release 2.0, I am -1 on this > merge until those blockers are addressed. > > I had a look at the list. > > I think the documentation issue is important but not actually a blocker. > That may be a controversial opinion, but documentation can be back-filled > worst case. So take HBASE-17133 off the list. > > Remaining are effectively HBASE-14417, HBASE-14141, and HBASE-15227. They > all have patches attached to the respective JIRAs so completing this work > won't be onerous. Get these committed and I will lift my -1. The others who > voted +1 on this thread surely can help with that. > > Thanks. > > > On Fri, Mar 10, 2017 at 2:32 PM, Vladimir Rodionov > > wrote: > > > No problem I will downgrade Blockers to Majors if it scares you, Andrew > 🙂 > > > > Sent from my iPhone > > > > > On Mar 10, 2017, at 1:52 PM, Andrew Purtell > wrote: > > > > > > I know the merge of this feature has lagged substantially. I think > that > > is > > > regrettable but on another thread we are lamenting that 2.0 is already > > > late. Unless I misunderstand, this is a proposal to merge something > with > > > known blockers into trunk before we branch it for 2.0 which will > > > effectively prevent that release because these blockers will be there. > I > > am > > > inclined to veto. Probably we should not propose branch merges into > code > > we > > > are trying to get out the door with known blockers. Why not do that > work > > > first? It seems an obvious question. Perhaps I am missing something. > > > > > > If we can branch for 2.0 now and then merge this, and not into the 2.0 > > > branch, I would vote +1 for branch merge even with known blockers > > pending. > > > > > > > > > On Fri, Mar 10, 2017 at 1:42 PM, Vladimir Rodionov < > > vladrodio...@gmail.com> > > > wrote: > > > > > >> They are not blockers for merge - only for 2.0. GA > > >> As I said already the feature is usable right now > > >> We would like to continue working on master and we would like to see a > > >> commitment from community > > >> > > >> Sent from my iPhone > > >> > > >> On Mar 10, 2017, at 11:16 AM, Andrew Purtell > > wrote: > > >> > > Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release. > > >>> > > >>> If we have identified blockers, why merge this before they are in? > > >>> Otherwise we can't release 2.0, and it is overdue. > > >>> > > >>> > > >>> On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov < > > >> vladrodio...@gmail.com> > > >>> wrote: > > >>> > > Hello, HBase folks > > > > For your consideration today is Backup/Restore feature for Apache > > HBAse > > 2.0. > > Backup code is available as a mega patch in HBASE-14123 (v61), > applies > > cleanly to the current master, all test PASS, patch has no other > > issues. > > > > The patch has gone through numerous rounds of code reviews and has > > >> probably > > the most lengthy discussion thread on Apache JIRA (HBASE-14123) :) > > > > The work has been split into 3 phases (HBASE-14030, 14123, 14414) > Two > > >> first > > are complete, third one is still in progress. > > > > > > *** Summary of work HBASE-14123 > > > > The new feature introduces new command-line extensions to the hbase > > >> command > > and, from the client side, is accessible through command-line only > > Operations: > > * Create full backup on a list of tables
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
Thanks for the offer but I like that you were honest about compiling a list of issues that you thought were blockers for release. Since this proposal is a merge into 2.0, and we are trying to release 2.0, I am -1 on this merge until those blockers are addressed. I had a look at the list. I think the documentation issue is important but not actually a blocker. That may be a controversial opinion, but documentation can be back-filled worst case. So take HBASE-17133 off the list. Remaining are effectively HBASE-14417, HBASE-14141, and HBASE-15227. They all have patches attached to the respective JIRAs so completing this work won't be onerous. Get these committed and I will lift my -1. The others who voted +1 on this thread surely can help with that. Thanks. On Fri, Mar 10, 2017 at 2:32 PM, Vladimir Rodionov wrote: > No problem I will downgrade Blockers to Majors if it scares you, Andrew 🙂 > > Sent from my iPhone > > > On Mar 10, 2017, at 1:52 PM, Andrew Purtell wrote: > > > > I know the merge of this feature has lagged substantially. I think that > is > > regrettable but on another thread we are lamenting that 2.0 is already > > late. Unless I misunderstand, this is a proposal to merge something with > > known blockers into trunk before we branch it for 2.0 which will > > effectively prevent that release because these blockers will be there. I > am > > inclined to veto. Probably we should not propose branch merges into code > we > > are trying to get out the door with known blockers. Why not do that work > > first? It seems an obvious question. Perhaps I am missing something. > > > > If we can branch for 2.0 now and then merge this, and not into the 2.0 > > branch, I would vote +1 for branch merge even with known blockers > pending. > > > > > > On Fri, Mar 10, 2017 at 1:42 PM, Vladimir Rodionov < > vladrodio...@gmail.com> > > wrote: > > > >> They are not blockers for merge - only for 2.0. GA > >> As I said already the feature is usable right now > >> We would like to continue working on master and we would like to see a > >> commitment from community > >> > >> Sent from my iPhone > >> > >> On Mar 10, 2017, at 11:16 AM, Andrew Purtell > wrote: > >> > Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release. > >>> > >>> If we have identified blockers, why merge this before they are in? > >>> Otherwise we can't release 2.0, and it is overdue. > >>> > >>> > >>> On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov < > >> vladrodio...@gmail.com> > >>> wrote: > >>> > Hello, HBase folks > > For your consideration today is Backup/Restore feature for Apache > HBAse > 2.0. > Backup code is available as a mega patch in HBASE-14123 (v61), applies > cleanly to the current master, all test PASS, patch has no other > issues. > > The patch has gone through numerous rounds of code reviews and has > >> probably > the most lengthy discussion thread on Apache JIRA (HBASE-14123) :) > > The work has been split into 3 phases (HBASE-14030, 14123, 14414) Two > >> first > are complete, third one is still in progress. > > > *** Summary of work HBASE-14123 > > The new feature introduces new command-line extensions to the hbase > >> command > and, from the client side, is accessible through command-line only > Operations: > * Create full backup on a list of tables or backup set > * Create incremental backup image for table list or backup set > * Restore list of tables from a given backup image > * Show current backup progress > * Delete backup image and all related images > * Show history of backups > * Backup set operations: create backup set, add/remove table to/from > >> backup > set, etc > > In the current implementation, the feature is already usable, meaning > >> that > users can backup tables and restore them using provided command-line > >> tools. > Both: full and incremental backups are supported. > This work is based on original work of IBM team (HBASE-7912). The full > >> list > of JIRAs included in this mega patch can be found in three umbrella > >> JIRAs: > HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 (Phase 3 > - > >> all > resolved ones made it into the patch) > > *** What are the remaining work items > > All remaining items can be found in Phase 3 umbrella JIRA: > HBASE-14414. > They are split into 3 groups: BLOCKER, CRITICAL, MAJOR > Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release. > > * BLOCKER > > * HBASE-14417 Incremental backup and bulk loading ( Patch available) > * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images > * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup to > include only edits from backup tables (Patch available) > * HBASE-17133 Backup documentation > * HBASE-15227 Fault t
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
No problem I will downgrade Blockers to Majors if it scares you, Andrew 🙂 Sent from my iPhone > On Mar 10, 2017, at 1:52 PM, Andrew Purtell wrote: > > I know the merge of this feature has lagged substantially. I think that is > regrettable but on another thread we are lamenting that 2.0 is already > late. Unless I misunderstand, this is a proposal to merge something with > known blockers into trunk before we branch it for 2.0 which will > effectively prevent that release because these blockers will be there. I am > inclined to veto. Probably we should not propose branch merges into code we > are trying to get out the door with known blockers. Why not do that work > first? It seems an obvious question. Perhaps I am missing something. > > If we can branch for 2.0 now and then merge this, and not into the 2.0 > branch, I would vote +1 for branch merge even with known blockers pending. > > > On Fri, Mar 10, 2017 at 1:42 PM, Vladimir Rodionov > wrote: > >> They are not blockers for merge - only for 2.0. GA >> As I said already the feature is usable right now >> We would like to continue working on master and we would like to see a >> commitment from community >> >> Sent from my iPhone >> >> On Mar 10, 2017, at 11:16 AM, Andrew Purtell wrote: >> Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release. >>> >>> If we have identified blockers, why merge this before they are in? >>> Otherwise we can't release 2.0, and it is overdue. >>> >>> >>> On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov < >> vladrodio...@gmail.com> >>> wrote: >>> Hello, HBase folks For your consideration today is Backup/Restore feature for Apache HBAse 2.0. Backup code is available as a mega patch in HBASE-14123 (v61), applies cleanly to the current master, all test PASS, patch has no other issues. The patch has gone through numerous rounds of code reviews and has >> probably the most lengthy discussion thread on Apache JIRA (HBASE-14123) :) The work has been split into 3 phases (HBASE-14030, 14123, 14414) Two >> first are complete, third one is still in progress. *** Summary of work HBASE-14123 The new feature introduces new command-line extensions to the hbase >> command and, from the client side, is accessible through command-line only Operations: * Create full backup on a list of tables or backup set * Create incremental backup image for table list or backup set * Restore list of tables from a given backup image * Show current backup progress * Delete backup image and all related images * Show history of backups * Backup set operations: create backup set, add/remove table to/from >> backup set, etc In the current implementation, the feature is already usable, meaning >> that users can backup tables and restore them using provided command-line >> tools. Both: full and incremental backups are supported. This work is based on original work of IBM team (HBASE-7912). The full >> list of JIRAs included in this mega patch can be found in three umbrella >> JIRAs: HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 (Phase 3 - >> all resolved ones made it into the patch) *** What are the remaining work items All remaining items can be found in Phase 3 umbrella JIRA: HBASE-14414. They are split into 3 groups: BLOCKER, CRITICAL, MAJOR Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release. * BLOCKER * HBASE-14417 Incremental backup and bulk loading ( Patch available) * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup to include only edits from backup tables (Patch available) * HBASE-17133 Backup documentation * HBASE-15227 Fault tolerance support * CRITICAL * HBASE-16465 Disable split/merges during backup We have umbrella JIRA (HBASE-14414) to track all the remaining work All the BLOCKER and CRITICAL JIRAs currently in open state will be implemented by 2.0 release time. Some MAJOR too, but it depends on >> resource availability The former development branch (HBASE-7912) is obsolete and will be closed/deleted after the merge. We want backup to be a GA feature in 2.0 We are going to support full backward compatibility for backup tool in >> 2.0 and onwards. Configuration Backup is disabled, by default. To enable it, the following >> configuration properties must be added to hbase-site.xml: hbase.backup.enable=true hbase.master.logcleaner.plugins=YOUR_PLUGINS,org. apache.hadoop.hbase.backup.master.BackupLogCleaner hbase.procedure.master.classes=YOUR_CLASSES,org. apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager >>>
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
I know the merge of this feature has lagged substantially. I think that is regrettable but on another thread we are lamenting that 2.0 is already late. Unless I misunderstand, this is a proposal to merge something with known blockers into trunk before we branch it for 2.0 which will effectively prevent that release because these blockers will be there. I am inclined to veto. Probably we should not propose branch merges into code we are trying to get out the door with known blockers. Why not do that work first? It seems an obvious question. Perhaps I am missing something. If we can branch for 2.0 now and then merge this, and not into the 2.0 branch, I would vote +1 for branch merge even with known blockers pending. On Fri, Mar 10, 2017 at 1:42 PM, Vladimir Rodionov wrote: > They are not blockers for merge - only for 2.0. GA > As I said already the feature is usable right now > We would like to continue working on master and we would like to see a > commitment from community > > Sent from my iPhone > > On Mar 10, 2017, at 11:16 AM, Andrew Purtell wrote: > > >> Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release. > > > > If we have identified blockers, why merge this before they are in? > > Otherwise we can't release 2.0, and it is overdue. > > > > > > On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov < > vladrodio...@gmail.com> > > wrote: > > > >> Hello, HBase folks > >> > >> For your consideration today is Backup/Restore feature for Apache HBAse > >> 2.0. > >> Backup code is available as a mega patch in HBASE-14123 (v61), applies > >> cleanly to the current master, all test PASS, patch has no other issues. > >> > >> The patch has gone through numerous rounds of code reviews and has > probably > >> the most lengthy discussion thread on Apache JIRA (HBASE-14123) :) > >> > >> The work has been split into 3 phases (HBASE-14030, 14123, 14414) Two > first > >> are complete, third one is still in progress. > >> > >> > >> *** Summary of work HBASE-14123 > >> > >> The new feature introduces new command-line extensions to the hbase > command > >> and, from the client side, is accessible through command-line only > >> Operations: > >> * Create full backup on a list of tables or backup set > >> * Create incremental backup image for table list or backup set > >> * Restore list of tables from a given backup image > >> * Show current backup progress > >> * Delete backup image and all related images > >> * Show history of backups > >> * Backup set operations: create backup set, add/remove table to/from > backup > >> set, etc > >> > >> In the current implementation, the feature is already usable, meaning > that > >> users can backup tables and restore them using provided command-line > tools. > >> Both: full and incremental backups are supported. > >> This work is based on original work of IBM team (HBASE-7912). The full > list > >> of JIRAs included in this mega patch can be found in three umbrella > JIRAs: > >> HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 (Phase 3 - > all > >> resolved ones made it into the patch) > >> > >> *** What are the remaining work items > >> > >> All remaining items can be found in Phase 3 umbrella JIRA: HBASE-14414. > >> They are split into 3 groups: BLOCKER, CRITICAL, MAJOR > >> Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release. > >> > >> * BLOCKER > >> > >> * HBASE-14417 Incremental backup and bulk loading ( Patch available) > >> * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images > >> * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup to > >> include only edits from backup tables (Patch available) > >> * HBASE-17133 Backup documentation > >> * HBASE-15227 Fault tolerance support > >> > >> * CRITICAL > >> > >> * HBASE-16465 Disable split/merges during backup > >> > >> We have umbrella JIRA (HBASE-14414) to track all the remaining work > >> All the BLOCKER and CRITICAL JIRAs currently in open state will be > >> implemented by 2.0 release time. Some MAJOR too, but it depends on > resource > >> availability > >> The former development branch (HBASE-7912) is obsolete and will be > >> closed/deleted after the merge. > >> We want backup to be a GA feature in 2.0 > >> We are going to support full backward compatibility for backup tool in > 2.0 > >> and onwards. > >> > >> Configuration > >> > >> Backup is disabled, by default. To enable it, the following > configuration > >> properties must be added to hbase-site.xml: > >> > >> hbase.backup.enable=true > >> hbase.master.logcleaner.plugins=YOUR_PLUGINS,org. > >> apache.hadoop.hbase.backup.master.BackupLogCleaner > >> hbase.procedure.master.classes=YOUR_CLASSES,org. > >> apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager > >> hbase.procedure.regionserver.classes=YOUR_CLASSES,org. > >> apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureMa > >> nager > >> > >> > >> I would like to thank IBM team and Jerry He for original work, > >> > >>
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
They are not blockers for merge - only for 2.0. GA As I said already the feature is usable right now We would like to continue working on master and we would like to see a commitment from community Sent from my iPhone On Mar 10, 2017, at 11:16 AM, Andrew Purtell wrote: >> Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release. > > If we have identified blockers, why merge this before they are in? > Otherwise we can't release 2.0, and it is overdue. > > > On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov > wrote: > >> Hello, HBase folks >> >> For your consideration today is Backup/Restore feature for Apache HBAse >> 2.0. >> Backup code is available as a mega patch in HBASE-14123 (v61), applies >> cleanly to the current master, all test PASS, patch has no other issues. >> >> The patch has gone through numerous rounds of code reviews and has probably >> the most lengthy discussion thread on Apache JIRA (HBASE-14123) :) >> >> The work has been split into 3 phases (HBASE-14030, 14123, 14414) Two first >> are complete, third one is still in progress. >> >> >> *** Summary of work HBASE-14123 >> >> The new feature introduces new command-line extensions to the hbase command >> and, from the client side, is accessible through command-line only >> Operations: >> * Create full backup on a list of tables or backup set >> * Create incremental backup image for table list or backup set >> * Restore list of tables from a given backup image >> * Show current backup progress >> * Delete backup image and all related images >> * Show history of backups >> * Backup set operations: create backup set, add/remove table to/from backup >> set, etc >> >> In the current implementation, the feature is already usable, meaning that >> users can backup tables and restore them using provided command-line tools. >> Both: full and incremental backups are supported. >> This work is based on original work of IBM team (HBASE-7912). The full list >> of JIRAs included in this mega patch can be found in three umbrella JIRAs: >> HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 (Phase 3 - all >> resolved ones made it into the patch) >> >> *** What are the remaining work items >> >> All remaining items can be found in Phase 3 umbrella JIRA: HBASE-14414. >> They are split into 3 groups: BLOCKER, CRITICAL, MAJOR >> Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release. >> >> * BLOCKER >> >> * HBASE-14417 Incremental backup and bulk loading ( Patch available) >> * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images >> * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup to >> include only edits from backup tables (Patch available) >> * HBASE-17133 Backup documentation >> * HBASE-15227 Fault tolerance support >> >> * CRITICAL >> >> * HBASE-16465 Disable split/merges during backup >> >> We have umbrella JIRA (HBASE-14414) to track all the remaining work >> All the BLOCKER and CRITICAL JIRAs currently in open state will be >> implemented by 2.0 release time. Some MAJOR too, but it depends on resource >> availability >> The former development branch (HBASE-7912) is obsolete and will be >> closed/deleted after the merge. >> We want backup to be a GA feature in 2.0 >> We are going to support full backward compatibility for backup tool in 2.0 >> and onwards. >> >> Configuration >> >> Backup is disabled, by default. To enable it, the following configuration >> properties must be added to hbase-site.xml: >> >> hbase.backup.enable=true >> hbase.master.logcleaner.plugins=YOUR_PLUGINS,org. >> apache.hadoop.hbase.backup.master.BackupLogCleaner >> hbase.procedure.master.classes=YOUR_CLASSES,org. >> apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager >> hbase.procedure.regionserver.classes=YOUR_CLASSES,org. >> apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureMa >> nager >> >> >> I would like to thank IBM team and Jerry He for original work, >> >> Enis, Ted, Stack, Matteo, Jerry for time spent on code reviews >> >> Special thanks to Ted Yu for his co-development work. >> >> References: >> >> https://issues.apache.org/jira/browse/HBASE-7912 (original IBM, contains >> design doc) >> https://issues.apache.org/jira/browse/HBASE-14030 (Phase 1) >> https://issues.apache.org/jira/browse/HBASE-14123 (Phase 2) >> https://issues.apache.org/jira/browse/HBASE-14414 (Phase 3) >> >> Please vote +1/-1 by midnight Pacific Time (00:00 >> -0800 GMT) on March 11th on whether or not we should merge this into the >> current master. >> >> -Vladimir Rodionov >> > > > > -- > Best regards, > > - Andy > > If you are given a choice, you believe you have acted freely. - Raymond > Teller (via Peter Watts)
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
> Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release. If we have identified blockers, why merge this before they are in? Otherwise we can't release 2.0, and it is overdue. On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov wrote: > Hello, HBase folks > > For your consideration today is Backup/Restore feature for Apache HBAse > 2.0. > Backup code is available as a mega patch in HBASE-14123 (v61), applies > cleanly to the current master, all test PASS, patch has no other issues. > > The patch has gone through numerous rounds of code reviews and has probably > the most lengthy discussion thread on Apache JIRA (HBASE-14123) :) > > The work has been split into 3 phases (HBASE-14030, 14123, 14414) Two first > are complete, third one is still in progress. > > > *** Summary of work HBASE-14123 > > The new feature introduces new command-line extensions to the hbase command > and, from the client side, is accessible through command-line only > Operations: > * Create full backup on a list of tables or backup set > * Create incremental backup image for table list or backup set > * Restore list of tables from a given backup image > * Show current backup progress > * Delete backup image and all related images > * Show history of backups > * Backup set operations: create backup set, add/remove table to/from backup > set, etc > > In the current implementation, the feature is already usable, meaning that > users can backup tables and restore them using provided command-line tools. > Both: full and incremental backups are supported. > This work is based on original work of IBM team (HBASE-7912). The full list > of JIRAs included in this mega patch can be found in three umbrella JIRAs: > HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 (Phase 3 - all > resolved ones made it into the patch) > > *** What are the remaining work items > > All remaining items can be found in Phase 3 umbrella JIRA: HBASE-14414. > They are split into 3 groups: BLOCKER, CRITICAL, MAJOR > Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release. > > * BLOCKER > > * HBASE-14417 Incremental backup and bulk loading ( Patch available) > * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images > * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup to > include only edits from backup tables (Patch available) > * HBASE-17133 Backup documentation > * HBASE-15227 Fault tolerance support > > * CRITICAL > > * HBASE-16465 Disable split/merges during backup > > We have umbrella JIRA (HBASE-14414) to track all the remaining work > All the BLOCKER and CRITICAL JIRAs currently in open state will be > implemented by 2.0 release time. Some MAJOR too, but it depends on resource > availability > The former development branch (HBASE-7912) is obsolete and will be > closed/deleted after the merge. > We want backup to be a GA feature in 2.0 > We are going to support full backward compatibility for backup tool in 2.0 > and onwards. > > Configuration > > Backup is disabled, by default. To enable it, the following configuration > properties must be added to hbase-site.xml: > > hbase.backup.enable=true > hbase.master.logcleaner.plugins=YOUR_PLUGINS,org. > apache.hadoop.hbase.backup.master.BackupLogCleaner > hbase.procedure.master.classes=YOUR_CLASSES,org. > apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager > hbase.procedure.regionserver.classes=YOUR_CLASSES,org. > apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureMa > nager > > > I would like to thank IBM team and Jerry He for original work, > > Enis, Ted, Stack, Matteo, Jerry for time spent on code reviews > > Special thanks to Ted Yu for his co-development work. > > References: > > https://issues.apache.org/jira/browse/HBASE-7912 (original IBM, contains > design doc) > https://issues.apache.org/jira/browse/HBASE-14030 (Phase 1) > https://issues.apache.org/jira/browse/HBASE-14123 (Phase 2) > https://issues.apache.org/jira/browse/HBASE-14414 (Phase 3) > > Please vote +1/-1 by midnight Pacific Time (00:00 > -0800 GMT) on March 11th on whether or not we should merge this into the > current master. > > -Vladimir Rodionov > -- Best regards, - Andy If you are given a choice, you believe you have acted freely. - Raymond Teller (via Peter Watts)
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
Still need one more +1 On Thu, Mar 9, 2017 at 11:27 PM, Vladimir Rodionov wrote: > bump > > On Thu, Mar 9, 2017 at 10:11 AM, Ted Yu wrote: > >> +1 from me as well. >> >> On Wed, Mar 8, 2017 at 3:48 PM, Enis Söztutar wrote: >> >> > Thanks Vladimir for the write up and the work. Glad to see progress. >> > >> > Here is my +1. I'm pretty sure we can get the blockers in before the 2.0 >> > timeframe with the momentum, so it is a good idea to merge now so that >> > development can continue in master, and there is more exposure for >> testing, >> > etc. >> > >> > Enis >> > >> > On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov < >> vladrodio...@gmail.com> >> > wrote: >> > >> > > Hello, HBase folks >> > > >> > > For your consideration today is Backup/Restore feature for Apache >> HBAse >> > > 2.0. >> > > Backup code is available as a mega patch in HBASE-14123 (v61), applies >> > > cleanly to the current master, all test PASS, patch has no other >> issues. >> > > >> > > The patch has gone through numerous rounds of code reviews and has >> > probably >> > > the most lengthy discussion thread on Apache JIRA (HBASE-14123) :) >> > > >> > > The work has been split into 3 phases (HBASE-14030, 14123, 14414) Two >> > first >> > > are complete, third one is still in progress. >> > > >> > > >> > > *** Summary of work HBASE-14123 >> > > >> > > The new feature introduces new command-line extensions to the hbase >> > command >> > > and, from the client side, is accessible through command-line only >> > > Operations: >> > > * Create full backup on a list of tables or backup set >> > > * Create incremental backup image for table list or backup set >> > > * Restore list of tables from a given backup image >> > > * Show current backup progress >> > > * Delete backup image and all related images >> > > * Show history of backups >> > > * Backup set operations: create backup set, add/remove table to/from >> > backup >> > > set, etc >> > > >> > > In the current implementation, the feature is already usable, meaning >> > that >> > > users can backup tables and restore them using provided command-line >> > tools. >> > > Both: full and incremental backups are supported. >> > > This work is based on original work of IBM team (HBASE-7912). The full >> > list >> > > of JIRAs included in this mega patch can be found in three umbrella >> > JIRAs: >> > > HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 (Phase 3 >> - >> > all >> > > resolved ones made it into the patch) >> > > >> > > *** What are the remaining work items >> > > >> > > All remaining items can be found in Phase 3 umbrella JIRA: >> HBASE-14414. >> > > They are split into 3 groups: BLOCKER, CRITICAL, MAJOR >> > > Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release. >> > > >> > > * BLOCKER >> > > >> > > * HBASE-14417 Incremental backup and bulk loading ( Patch available) >> > > * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images >> > > * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup to >> > > include only edits from backup tables (Patch available) >> > > * HBASE-17133 Backup documentation >> > > * HBASE-15227 Fault tolerance support >> > > >> > > * CRITICAL >> > > >> > > * HBASE-16465 Disable split/merges during backup >> > > >> > > We have umbrella JIRA (HBASE-14414) to track all the remaining work >> > > All the BLOCKER and CRITICAL JIRAs currently in open state will be >> > > implemented by 2.0 release time. Some MAJOR too, but it depends on >> > resource >> > > availability >> > > The former development branch (HBASE-7912) is obsolete and will be >> > > closed/deleted after the merge. >> > > We want backup to be a GA feature in 2.0 >> > > We are going to support full backward compatibility for backup tool in >> > 2.0 >> > > and onwards. >> > > >> > > Configuration >> > > >> > > Backup is disabled, by default. To enable it, the following >> configuration >> > > properties must be added to hbase-site.xml: >> > > >> > > hbase.backup.enable=true >> > > hbase.master.logcleaner.plugins=YOUR_PLUGINS,org. >> > > apache.hadoop.hbase.backup.master.BackupLogCleaner >> > > hbase.procedure.master.classes=YOUR_CLASSES,org. >> > > apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager >> > > hbase.procedure.regionserver.classes=YOUR_CLASSES,org. >> > > apache.hadoop.hbase.backup.regionserver.LogRollRegionServerP >> rocedureMa >> > > nager >> > > >> > > >> > > I would like to thank IBM team and Jerry He for original work, >> > > >> > > Enis, Ted, Stack, Matteo, Jerry for time spent on code reviews >> > > >> > > Special thanks to Ted Yu for his co-development work. >> > > >> > > References: >> > > >> > > https://issues.apache.org/jira/browse/HBASE-7912 (original IBM, >> contains >> > > design doc) >> > > https://issues.apache.org/jira/browse/HBASE-14030 (Phase 1) >> > > https://issues.apache.org/jira/browse/HBASE-14123 (Phase 2) >> > > https://issues.apache.org/jira/browse/HBASE-14414 (Phase 3) >> > > >> > > Please
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
bump On Thu, Mar 9, 2017 at 10:11 AM, Ted Yu wrote: > +1 from me as well. > > On Wed, Mar 8, 2017 at 3:48 PM, Enis Söztutar wrote: > > > Thanks Vladimir for the write up and the work. Glad to see progress. > > > > Here is my +1. I'm pretty sure we can get the blockers in before the 2.0 > > timeframe with the momentum, so it is a good idea to merge now so that > > development can continue in master, and there is more exposure for > testing, > > etc. > > > > Enis > > > > On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov < > vladrodio...@gmail.com> > > wrote: > > > > > Hello, HBase folks > > > > > > For your consideration today is Backup/Restore feature for Apache HBAse > > > 2.0. > > > Backup code is available as a mega patch in HBASE-14123 (v61), applies > > > cleanly to the current master, all test PASS, patch has no other > issues. > > > > > > The patch has gone through numerous rounds of code reviews and has > > probably > > > the most lengthy discussion thread on Apache JIRA (HBASE-14123) :) > > > > > > The work has been split into 3 phases (HBASE-14030, 14123, 14414) Two > > first > > > are complete, third one is still in progress. > > > > > > > > > *** Summary of work HBASE-14123 > > > > > > The new feature introduces new command-line extensions to the hbase > > command > > > and, from the client side, is accessible through command-line only > > > Operations: > > > * Create full backup on a list of tables or backup set > > > * Create incremental backup image for table list or backup set > > > * Restore list of tables from a given backup image > > > * Show current backup progress > > > * Delete backup image and all related images > > > * Show history of backups > > > * Backup set operations: create backup set, add/remove table to/from > > backup > > > set, etc > > > > > > In the current implementation, the feature is already usable, meaning > > that > > > users can backup tables and restore them using provided command-line > > tools. > > > Both: full and incremental backups are supported. > > > This work is based on original work of IBM team (HBASE-7912). The full > > list > > > of JIRAs included in this mega patch can be found in three umbrella > > JIRAs: > > > HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 (Phase 3 - > > all > > > resolved ones made it into the patch) > > > > > > *** What are the remaining work items > > > > > > All remaining items can be found in Phase 3 umbrella JIRA: HBASE-14414. > > > They are split into 3 groups: BLOCKER, CRITICAL, MAJOR > > > Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release. > > > > > > * BLOCKER > > > > > > * HBASE-14417 Incremental backup and bulk loading ( Patch available) > > > * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images > > > * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup to > > > include only edits from backup tables (Patch available) > > > * HBASE-17133 Backup documentation > > > * HBASE-15227 Fault tolerance support > > > > > > * CRITICAL > > > > > > * HBASE-16465 Disable split/merges during backup > > > > > > We have umbrella JIRA (HBASE-14414) to track all the remaining work > > > All the BLOCKER and CRITICAL JIRAs currently in open state will be > > > implemented by 2.0 release time. Some MAJOR too, but it depends on > > resource > > > availability > > > The former development branch (HBASE-7912) is obsolete and will be > > > closed/deleted after the merge. > > > We want backup to be a GA feature in 2.0 > > > We are going to support full backward compatibility for backup tool in > > 2.0 > > > and onwards. > > > > > > Configuration > > > > > > Backup is disabled, by default. To enable it, the following > configuration > > > properties must be added to hbase-site.xml: > > > > > > hbase.backup.enable=true > > > hbase.master.logcleaner.plugins=YOUR_PLUGINS,org. > > > apache.hadoop.hbase.backup.master.BackupLogCleaner > > > hbase.procedure.master.classes=YOUR_CLASSES,org. > > > apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager > > > hbase.procedure.regionserver.classes=YOUR_CLASSES,org. > > > apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureMa > > > nager > > > > > > > > > I would like to thank IBM team and Jerry He for original work, > > > > > > Enis, Ted, Stack, Matteo, Jerry for time spent on code reviews > > > > > > Special thanks to Ted Yu for his co-development work. > > > > > > References: > > > > > > https://issues.apache.org/jira/browse/HBASE-7912 (original IBM, > contains > > > design doc) > > > https://issues.apache.org/jira/browse/HBASE-14030 (Phase 1) > > > https://issues.apache.org/jira/browse/HBASE-14123 (Phase 2) > > > https://issues.apache.org/jira/browse/HBASE-14414 (Phase 3) > > > > > > Please vote +1/-1 by midnight Pacific Time (00:00 > > > -0800 GMT) on March 11th on whether or not we should merge this into > > the > > > current master. > > > > > > -Vladimir Rodionov > > > > > >
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
bump On Wed, Mar 8, 2017 at 3:52 PM, Vladimir Rodionov wrote: > No problem, we can extend deadline. > > On Wed, Mar 8, 2017 at 3:31 PM, Ted Yu wrote: > >> March 11th is on weekend. >> >> Do you want to give people who haven't looked at the mega patch in depth >> some more time ? >> >> Cheers >> >> On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov > > >> wrote: >> >> > Hello, HBase folks >> > >> > For your consideration today is Backup/Restore feature for Apache HBAse >> > 2.0. >> > Backup code is available as a mega patch in HBASE-14123 (v61), applies >> > cleanly to the current master, all test PASS, patch has no other issues. >> > >> > The patch has gone through numerous rounds of code reviews and has >> probably >> > the most lengthy discussion thread on Apache JIRA (HBASE-14123) :) >> > >> > The work has been split into 3 phases (HBASE-14030, 14123, 14414) Two >> first >> > are complete, third one is still in progress. >> > >> > >> > *** Summary of work HBASE-14123 >> > >> > The new feature introduces new command-line extensions to the hbase >> command >> > and, from the client side, is accessible through command-line only >> > Operations: >> > * Create full backup on a list of tables or backup set >> > * Create incremental backup image for table list or backup set >> > * Restore list of tables from a given backup image >> > * Show current backup progress >> > * Delete backup image and all related images >> > * Show history of backups >> > * Backup set operations: create backup set, add/remove table to/from >> backup >> > set, etc >> > >> > In the current implementation, the feature is already usable, meaning >> that >> > users can backup tables and restore them using provided command-line >> tools. >> > Both: full and incremental backups are supported. >> > This work is based on original work of IBM team (HBASE-7912). The full >> list >> > of JIRAs included in this mega patch can be found in three umbrella >> JIRAs: >> > HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 (Phase 3 - >> all >> > resolved ones made it into the patch) >> > >> > *** What are the remaining work items >> > >> > All remaining items can be found in Phase 3 umbrella JIRA: HBASE-14414. >> > They are split into 3 groups: BLOCKER, CRITICAL, MAJOR >> > Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release. >> > >> > * BLOCKER >> > >> > * HBASE-14417 Incremental backup and bulk loading ( Patch available) >> > * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images >> > * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup to >> > include only edits from backup tables (Patch available) >> > * HBASE-17133 Backup documentation >> > * HBASE-15227 Fault tolerance support >> > >> > * CRITICAL >> > >> > * HBASE-16465 Disable split/merges during backup >> > >> > We have umbrella JIRA (HBASE-14414) to track all the remaining work >> > All the BLOCKER and CRITICAL JIRAs currently in open state will be >> > implemented by 2.0 release time. Some MAJOR too, but it depends on >> resource >> > availability >> > The former development branch (HBASE-7912) is obsolete and will be >> > closed/deleted after the merge. >> > We want backup to be a GA feature in 2.0 >> > We are going to support full backward compatibility for backup tool in >> 2.0 >> > and onwards. >> > >> > Configuration >> > >> > Backup is disabled, by default. To enable it, the following >> configuration >> > properties must be added to hbase-site.xml: >> > >> > hbase.backup.enable=true >> > hbase.master.logcleaner.plugins=YOUR_PLUGINS,org. >> > apache.hadoop.hbase.backup.master.BackupLogCleaner >> > hbase.procedure.master.classes=YOUR_CLASSES,org. >> > apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager >> > hbase.procedure.regionserver.classes=YOUR_CLASSES,org. >> > apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureMa >> > nager >> > >> > >> > I would like to thank IBM team and Jerry He for original work, >> > >> > Enis, Ted, Stack, Matteo, Jerry for time spent on code reviews >> > >> > Special thanks to Ted Yu for his co-development work. >> > >> > References: >> > >> > https://issues.apache.org/jira/browse/HBASE-7912 (original IBM, >> contains >> > design doc) >> > https://issues.apache.org/jira/browse/HBASE-14030 (Phase 1) >> > https://issues.apache.org/jira/browse/HBASE-14123 (Phase 2) >> > https://issues.apache.org/jira/browse/HBASE-14414 (Phase 3) >> > >> > Please vote +1/-1 by midnight Pacific Time (00:00 >> > -0800 GMT) on March 11th on whether or not we should merge this into >> the >> > current master. >> > >> > -Vladimir Rodionov >> > >> > >
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
+1 from me as well. On Wed, Mar 8, 2017 at 3:48 PM, Enis Söztutar wrote: > Thanks Vladimir for the write up and the work. Glad to see progress. > > Here is my +1. I'm pretty sure we can get the blockers in before the 2.0 > timeframe with the momentum, so it is a good idea to merge now so that > development can continue in master, and there is more exposure for testing, > etc. > > Enis > > On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov > wrote: > > > Hello, HBase folks > > > > For your consideration today is Backup/Restore feature for Apache HBAse > > 2.0. > > Backup code is available as a mega patch in HBASE-14123 (v61), applies > > cleanly to the current master, all test PASS, patch has no other issues. > > > > The patch has gone through numerous rounds of code reviews and has > probably > > the most lengthy discussion thread on Apache JIRA (HBASE-14123) :) > > > > The work has been split into 3 phases (HBASE-14030, 14123, 14414) Two > first > > are complete, third one is still in progress. > > > > > > *** Summary of work HBASE-14123 > > > > The new feature introduces new command-line extensions to the hbase > command > > and, from the client side, is accessible through command-line only > > Operations: > > * Create full backup on a list of tables or backup set > > * Create incremental backup image for table list or backup set > > * Restore list of tables from a given backup image > > * Show current backup progress > > * Delete backup image and all related images > > * Show history of backups > > * Backup set operations: create backup set, add/remove table to/from > backup > > set, etc > > > > In the current implementation, the feature is already usable, meaning > that > > users can backup tables and restore them using provided command-line > tools. > > Both: full and incremental backups are supported. > > This work is based on original work of IBM team (HBASE-7912). The full > list > > of JIRAs included in this mega patch can be found in three umbrella > JIRAs: > > HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 (Phase 3 - > all > > resolved ones made it into the patch) > > > > *** What are the remaining work items > > > > All remaining items can be found in Phase 3 umbrella JIRA: HBASE-14414. > > They are split into 3 groups: BLOCKER, CRITICAL, MAJOR > > Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release. > > > > * BLOCKER > > > > * HBASE-14417 Incremental backup and bulk loading ( Patch available) > > * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images > > * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup to > > include only edits from backup tables (Patch available) > > * HBASE-17133 Backup documentation > > * HBASE-15227 Fault tolerance support > > > > * CRITICAL > > > > * HBASE-16465 Disable split/merges during backup > > > > We have umbrella JIRA (HBASE-14414) to track all the remaining work > > All the BLOCKER and CRITICAL JIRAs currently in open state will be > > implemented by 2.0 release time. Some MAJOR too, but it depends on > resource > > availability > > The former development branch (HBASE-7912) is obsolete and will be > > closed/deleted after the merge. > > We want backup to be a GA feature in 2.0 > > We are going to support full backward compatibility for backup tool in > 2.0 > > and onwards. > > > > Configuration > > > > Backup is disabled, by default. To enable it, the following configuration > > properties must be added to hbase-site.xml: > > > > hbase.backup.enable=true > > hbase.master.logcleaner.plugins=YOUR_PLUGINS,org. > > apache.hadoop.hbase.backup.master.BackupLogCleaner > > hbase.procedure.master.classes=YOUR_CLASSES,org. > > apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager > > hbase.procedure.regionserver.classes=YOUR_CLASSES,org. > > apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureMa > > nager > > > > > > I would like to thank IBM team and Jerry He for original work, > > > > Enis, Ted, Stack, Matteo, Jerry for time spent on code reviews > > > > Special thanks to Ted Yu for his co-development work. > > > > References: > > > > https://issues.apache.org/jira/browse/HBASE-7912 (original IBM, contains > > design doc) > > https://issues.apache.org/jira/browse/HBASE-14030 (Phase 1) > > https://issues.apache.org/jira/browse/HBASE-14123 (Phase 2) > > https://issues.apache.org/jira/browse/HBASE-14414 (Phase 3) > > > > Please vote +1/-1 by midnight Pacific Time (00:00 > > -0800 GMT) on March 11th on whether or not we should merge this into > the > > current master. > > > > -Vladimir Rodionov > > >
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
No problem, we can extend deadline. On Wed, Mar 8, 2017 at 3:31 PM, Ted Yu wrote: > March 11th is on weekend. > > Do you want to give people who haven't looked at the mega patch in depth > some more time ? > > Cheers > > On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov > wrote: > > > Hello, HBase folks > > > > For your consideration today is Backup/Restore feature for Apache HBAse > > 2.0. > > Backup code is available as a mega patch in HBASE-14123 (v61), applies > > cleanly to the current master, all test PASS, patch has no other issues. > > > > The patch has gone through numerous rounds of code reviews and has > probably > > the most lengthy discussion thread on Apache JIRA (HBASE-14123) :) > > > > The work has been split into 3 phases (HBASE-14030, 14123, 14414) Two > first > > are complete, third one is still in progress. > > > > > > *** Summary of work HBASE-14123 > > > > The new feature introduces new command-line extensions to the hbase > command > > and, from the client side, is accessible through command-line only > > Operations: > > * Create full backup on a list of tables or backup set > > * Create incremental backup image for table list or backup set > > * Restore list of tables from a given backup image > > * Show current backup progress > > * Delete backup image and all related images > > * Show history of backups > > * Backup set operations: create backup set, add/remove table to/from > backup > > set, etc > > > > In the current implementation, the feature is already usable, meaning > that > > users can backup tables and restore them using provided command-line > tools. > > Both: full and incremental backups are supported. > > This work is based on original work of IBM team (HBASE-7912). The full > list > > of JIRAs included in this mega patch can be found in three umbrella > JIRAs: > > HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 (Phase 3 - > all > > resolved ones made it into the patch) > > > > *** What are the remaining work items > > > > All remaining items can be found in Phase 3 umbrella JIRA: HBASE-14414. > > They are split into 3 groups: BLOCKER, CRITICAL, MAJOR > > Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release. > > > > * BLOCKER > > > > * HBASE-14417 Incremental backup and bulk loading ( Patch available) > > * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images > > * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup to > > include only edits from backup tables (Patch available) > > * HBASE-17133 Backup documentation > > * HBASE-15227 Fault tolerance support > > > > * CRITICAL > > > > * HBASE-16465 Disable split/merges during backup > > > > We have umbrella JIRA (HBASE-14414) to track all the remaining work > > All the BLOCKER and CRITICAL JIRAs currently in open state will be > > implemented by 2.0 release time. Some MAJOR too, but it depends on > resource > > availability > > The former development branch (HBASE-7912) is obsolete and will be > > closed/deleted after the merge. > > We want backup to be a GA feature in 2.0 > > We are going to support full backward compatibility for backup tool in > 2.0 > > and onwards. > > > > Configuration > > > > Backup is disabled, by default. To enable it, the following configuration > > properties must be added to hbase-site.xml: > > > > hbase.backup.enable=true > > hbase.master.logcleaner.plugins=YOUR_PLUGINS,org. > > apache.hadoop.hbase.backup.master.BackupLogCleaner > > hbase.procedure.master.classes=YOUR_CLASSES,org. > > apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager > > hbase.procedure.regionserver.classes=YOUR_CLASSES,org. > > apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureMa > > nager > > > > > > I would like to thank IBM team and Jerry He for original work, > > > > Enis, Ted, Stack, Matteo, Jerry for time spent on code reviews > > > > Special thanks to Ted Yu for his co-development work. > > > > References: > > > > https://issues.apache.org/jira/browse/HBASE-7912 (original IBM, contains > > design doc) > > https://issues.apache.org/jira/browse/HBASE-14030 (Phase 1) > > https://issues.apache.org/jira/browse/HBASE-14123 (Phase 2) > > https://issues.apache.org/jira/browse/HBASE-14414 (Phase 3) > > > > Please vote +1/-1 by midnight Pacific Time (00:00 > > -0800 GMT) on March 11th on whether or not we should merge this into > the > > current master. > > > > -Vladimir Rodionov > > >
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
Thanks Vladimir for the write up and the work. Glad to see progress. Here is my +1. I'm pretty sure we can get the blockers in before the 2.0 timeframe with the momentum, so it is a good idea to merge now so that development can continue in master, and there is more exposure for testing, etc. Enis On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov wrote: > Hello, HBase folks > > For your consideration today is Backup/Restore feature for Apache HBAse > 2.0. > Backup code is available as a mega patch in HBASE-14123 (v61), applies > cleanly to the current master, all test PASS, patch has no other issues. > > The patch has gone through numerous rounds of code reviews and has probably > the most lengthy discussion thread on Apache JIRA (HBASE-14123) :) > > The work has been split into 3 phases (HBASE-14030, 14123, 14414) Two first > are complete, third one is still in progress. > > > *** Summary of work HBASE-14123 > > The new feature introduces new command-line extensions to the hbase command > and, from the client side, is accessible through command-line only > Operations: > * Create full backup on a list of tables or backup set > * Create incremental backup image for table list or backup set > * Restore list of tables from a given backup image > * Show current backup progress > * Delete backup image and all related images > * Show history of backups > * Backup set operations: create backup set, add/remove table to/from backup > set, etc > > In the current implementation, the feature is already usable, meaning that > users can backup tables and restore them using provided command-line tools. > Both: full and incremental backups are supported. > This work is based on original work of IBM team (HBASE-7912). The full list > of JIRAs included in this mega patch can be found in three umbrella JIRAs: > HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 (Phase 3 - all > resolved ones made it into the patch) > > *** What are the remaining work items > > All remaining items can be found in Phase 3 umbrella JIRA: HBASE-14414. > They are split into 3 groups: BLOCKER, CRITICAL, MAJOR > Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release. > > * BLOCKER > > * HBASE-14417 Incremental backup and bulk loading ( Patch available) > * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images > * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup to > include only edits from backup tables (Patch available) > * HBASE-17133 Backup documentation > * HBASE-15227 Fault tolerance support > > * CRITICAL > > * HBASE-16465 Disable split/merges during backup > > We have umbrella JIRA (HBASE-14414) to track all the remaining work > All the BLOCKER and CRITICAL JIRAs currently in open state will be > implemented by 2.0 release time. Some MAJOR too, but it depends on resource > availability > The former development branch (HBASE-7912) is obsolete and will be > closed/deleted after the merge. > We want backup to be a GA feature in 2.0 > We are going to support full backward compatibility for backup tool in 2.0 > and onwards. > > Configuration > > Backup is disabled, by default. To enable it, the following configuration > properties must be added to hbase-site.xml: > > hbase.backup.enable=true > hbase.master.logcleaner.plugins=YOUR_PLUGINS,org. > apache.hadoop.hbase.backup.master.BackupLogCleaner > hbase.procedure.master.classes=YOUR_CLASSES,org. > apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager > hbase.procedure.regionserver.classes=YOUR_CLASSES,org. > apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureMa > nager > > > I would like to thank IBM team and Jerry He for original work, > > Enis, Ted, Stack, Matteo, Jerry for time spent on code reviews > > Special thanks to Ted Yu for his co-development work. > > References: > > https://issues.apache.org/jira/browse/HBASE-7912 (original IBM, contains > design doc) > https://issues.apache.org/jira/browse/HBASE-14030 (Phase 1) > https://issues.apache.org/jira/browse/HBASE-14123 (Phase 2) > https://issues.apache.org/jira/browse/HBASE-14414 (Phase 3) > > Please vote +1/-1 by midnight Pacific Time (00:00 > -0800 GMT) on March 11th on whether or not we should merge this into the > current master. > > -Vladimir Rodionov >
Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
March 11th is on weekend. Do you want to give people who haven't looked at the mega patch in depth some more time ? Cheers On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov wrote: > Hello, HBase folks > > For your consideration today is Backup/Restore feature for Apache HBAse > 2.0. > Backup code is available as a mega patch in HBASE-14123 (v61), applies > cleanly to the current master, all test PASS, patch has no other issues. > > The patch has gone through numerous rounds of code reviews and has probably > the most lengthy discussion thread on Apache JIRA (HBASE-14123) :) > > The work has been split into 3 phases (HBASE-14030, 14123, 14414) Two first > are complete, third one is still in progress. > > > *** Summary of work HBASE-14123 > > The new feature introduces new command-line extensions to the hbase command > and, from the client side, is accessible through command-line only > Operations: > * Create full backup on a list of tables or backup set > * Create incremental backup image for table list or backup set > * Restore list of tables from a given backup image > * Show current backup progress > * Delete backup image and all related images > * Show history of backups > * Backup set operations: create backup set, add/remove table to/from backup > set, etc > > In the current implementation, the feature is already usable, meaning that > users can backup tables and restore them using provided command-line tools. > Both: full and incremental backups are supported. > This work is based on original work of IBM team (HBASE-7912). The full list > of JIRAs included in this mega patch can be found in three umbrella JIRAs: > HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 (Phase 3 - all > resolved ones made it into the patch) > > *** What are the remaining work items > > All remaining items can be found in Phase 3 umbrella JIRA: HBASE-14414. > They are split into 3 groups: BLOCKER, CRITICAL, MAJOR > Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release. > > * BLOCKER > > * HBASE-14417 Incremental backup and bulk loading ( Patch available) > * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images > * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup to > include only edits from backup tables (Patch available) > * HBASE-17133 Backup documentation > * HBASE-15227 Fault tolerance support > > * CRITICAL > > * HBASE-16465 Disable split/merges during backup > > We have umbrella JIRA (HBASE-14414) to track all the remaining work > All the BLOCKER and CRITICAL JIRAs currently in open state will be > implemented by 2.0 release time. Some MAJOR too, but it depends on resource > availability > The former development branch (HBASE-7912) is obsolete and will be > closed/deleted after the merge. > We want backup to be a GA feature in 2.0 > We are going to support full backward compatibility for backup tool in 2.0 > and onwards. > > Configuration > > Backup is disabled, by default. To enable it, the following configuration > properties must be added to hbase-site.xml: > > hbase.backup.enable=true > hbase.master.logcleaner.plugins=YOUR_PLUGINS,org. > apache.hadoop.hbase.backup.master.BackupLogCleaner > hbase.procedure.master.classes=YOUR_CLASSES,org. > apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager > hbase.procedure.regionserver.classes=YOUR_CLASSES,org. > apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureMa > nager > > > I would like to thank IBM team and Jerry He for original work, > > Enis, Ted, Stack, Matteo, Jerry for time spent on code reviews > > Special thanks to Ted Yu for his co-development work. > > References: > > https://issues.apache.org/jira/browse/HBASE-7912 (original IBM, contains > design doc) > https://issues.apache.org/jira/browse/HBASE-14030 (Phase 1) > https://issues.apache.org/jira/browse/HBASE-14123 (Phase 2) > https://issues.apache.org/jira/browse/HBASE-14414 (Phase 3) > > Please vote +1/-1 by midnight Pacific Time (00:00 > -0800 GMT) on March 11th on whether or not we should merge this into the > current master. > > -Vladimir Rodionov >
[VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
Hello, HBase folks For your consideration today is Backup/Restore feature for Apache HBAse 2.0. Backup code is available as a mega patch in HBASE-14123 (v61), applies cleanly to the current master, all test PASS, patch has no other issues. The patch has gone through numerous rounds of code reviews and has probably the most lengthy discussion thread on Apache JIRA (HBASE-14123) :) The work has been split into 3 phases (HBASE-14030, 14123, 14414) Two first are complete, third one is still in progress. *** Summary of work HBASE-14123 The new feature introduces new command-line extensions to the hbase command and, from the client side, is accessible through command-line only Operations: * Create full backup on a list of tables or backup set * Create incremental backup image for table list or backup set * Restore list of tables from a given backup image * Show current backup progress * Delete backup image and all related images * Show history of backups * Backup set operations: create backup set, add/remove table to/from backup set, etc In the current implementation, the feature is already usable, meaning that users can backup tables and restore them using provided command-line tools. Both: full and incremental backups are supported. This work is based on original work of IBM team (HBASE-7912). The full list of JIRAs included in this mega patch can be found in three umbrella JIRAs: HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 (Phase 3 - all resolved ones made it into the patch) *** What are the remaining work items All remaining items can be found in Phase 3 umbrella JIRA: HBASE-14414. They are split into 3 groups: BLOCKER, CRITICAL, MAJOR Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release. * BLOCKER * HBASE-14417 Incremental backup and bulk loading ( Patch available) * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup to include only edits from backup tables (Patch available) * HBASE-17133 Backup documentation * HBASE-15227 Fault tolerance support * CRITICAL * HBASE-16465 Disable split/merges during backup We have umbrella JIRA (HBASE-14414) to track all the remaining work All the BLOCKER and CRITICAL JIRAs currently in open state will be implemented by 2.0 release time. Some MAJOR too, but it depends on resource availability The former development branch (HBASE-7912) is obsolete and will be closed/deleted after the merge. We want backup to be a GA feature in 2.0 We are going to support full backward compatibility for backup tool in 2.0 and onwards. Configuration Backup is disabled, by default. To enable it, the following configuration properties must be added to hbase-site.xml: hbase.backup.enable=true hbase.master.logcleaner.plugins=YOUR_PLUGINS,org.apache.hadoop.hbase.backup.master.BackupLogCleaner hbase.procedure.master.classes=YOUR_CLASSES,org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager hbase.procedure.regionserver.classes=YOUR_CLASSES,org.apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureManager I would like to thank IBM team and Jerry He for original work, Enis, Ted, Stack, Matteo, Jerry for time spent on code reviews Special thanks to Ted Yu for his co-development work. References: https://issues.apache.org/jira/browse/HBASE-7912 (original IBM, contains design doc) https://issues.apache.org/jira/browse/HBASE-14030 (Phase 1) https://issues.apache.org/jira/browse/HBASE-14123 (Phase 2) https://issues.apache.org/jira/browse/HBASE-14414 (Phase 3) Please vote +1/-1 by midnight Pacific Time (00:00 -0800 GMT) on March 11th on whether or not we should merge this into the current master. -Vladimir Rodionov