Jon: Mind taking a look at this attachment for list of commits (in case I missed any):
https://issues.apache.org/jira/secure/attachment/12791047/mob-cmmits.txt Thanks On Wed, Mar 2, 2016 at 7:26 PM, Jonathan Hsieh <[email protected]> wrote: > Most of the patches should have some mention of "mob" in their titles. > Another sure sign is to look for commits attributed to jingcheng du -- he > contributed the majority of the patches. I did a few mostly adding flags > for testing and that bulk load related bug fix. > > I'll review the list of patches as part of any branch merge proposal to > make sure all the necessary ones are present. > > On more minor thing -- instead of the longish hbase-11339-branch-1 name I > proposed -- naming the branch the name umbrella jira of the backport > (hbase-15370) would make it even easier to track provenance of the branch. > > Jon. > > On Wed, Mar 2, 2016 at 6:44 PM, Devaraj Das <[email protected]> wrote: > > > Sounds good to me, Jonathan. Although I suspect that at the end this will > > be close to what Ted Yu has in the backport patch but this is a more > > structured / reliable way of doing it. When you say this - "backport the > > patches that came after the merge to trunk....", is there a reliable way > to > > identify all those. In the backport jira, there is a mention of some > > additional jiras after the original merge - https://goo.gl/6stN7Y - we > > need to make sure if these are the only ones relevant. > > ________________________________________ > > From: Jonathan Hsieh <[email protected]> > > Sent: Wednesday, March 02, 2016 2:03 PM > > To: [email protected] > > Subject: Re: MOB in branch-1? (Re: [RESULT][VOTE] Merge branch > hbase-11339 > > HBase MOB to trunk) > > > > I'll offer what I think would be another reasonable and likely less > > destabilizing approach: essentially treat it as a branch. We could > start > > from the existing hbase-11339 branch, call it hbase-11339-branch-1 and do > > the work to merge it into branch-1 (instead of trunk). To merge into > > branch-1 it would require the 3 +1's just like any other branch merge. > > > > Mob initially started off as a mega patch and was broken down and > committed > > in reviewable pieces to branch hbase-11339. Most of this was actually > when > > 1.0.0-SNAPSHOT was trunk so many of the older commits may actually jive > > better with branch-1. Towards the end we did periodic merge commits with > > trunk as we shook out issues integration testing found, and as branch > merge > > reviews uncovered other short comings. These merge patches actually > showed > > how we modified the mob code to account for the later changes in trunk. > In > > this particular case we coud take hbase-11339-branch-1, backport the > > patches that came after the merge to trunk with the normal single +1 > review > > process on the branch, and then propose a branch merge with branch-1. > When > > all pieces are ready and we had stable unit tests (admittedly that was a > > one shortcoming when I sheparded hbase-11339) we do 3 +1s to merge to > > branch-1 (and possibly branch-1.3). > > > > This has the nice benefit compared to andrew's suggesting of not > > potentially blocking branch-1 or having a half complete mob feature in > > branch-1 if hbase-11339-branch-1 isn't ready. > > > > Jon. > > > > > > > > > > On Wed, Mar 2, 2016 at 1:31 PM, Andrew Purtell <[email protected] > > > > wrote: > > > > > On the question of change provenance, if I can make a suggestion: How I > > > would approach the backport of a feature developed in trunk over > multiple > > > commits and JIRAs is I would look through git history, find each > related > > > commit, and apply each commit in sequence to branch-1, fixing things up > > as > > > needed each time. Then the resulting working branch can be pushed up to > > aid > > > review of the aggregate result by the MOB developers and the larger > > > community. Ted, if you approached the backport in this way, then > pushing > > up > > > your working branch will aid review. > > > > > > > On Mar 2, 2016, at 1:17 PM, Andrew Purtell <[email protected] > > > > > wrote: > > > > > > > > I concur with Stack's concerns with a developer not involved in > > building > > > the MOB feature attempting a backport in general, and furthermore > without > > > detailed provenance of all of the incorporated changes. I'm also > > concerned > > > about how well MOB works in real production given people use branch-1 > in > > > production and not trunk. > > > > > > > > As for Ted's issue, I'm not going to let him commit it (I will veto) > > > without clear proof that it works and doesn't hurt perf of non MOB > cases > > by > > > way of reproducible benchmarks - benchmarks I can personally reproduce > > > afterward (I'm not volunteering to do Ted's necessary legwork) - done > > with > > > branch-1 with the proposed patch applied. We are not there yet though, > > > barely, into review, but this will be coming up very soon. > > > > > > > >>> On Mar 2, 2016, at 1:03 PM, Stack <[email protected]> wrote: > > > >>> > > > >>> On Wed, Mar 2, 2016 at 11:26 AM, Devaraj Das <[email protected] > > > > > wrote: > > > >>> > > > >>> Stack, as things stand, Ted Yu has a patch that is a backport of > MOB. > > > He > > > >>> told me offline he has taken a look at the jiras that went in in > the > > > master > > > >>> that is to do with MOB, and got them in the backport. > > > >> > > > >> > > > >> > > > >> On the issue, a near 800k blob is dumped which is described as "...a > > > >> backport of MOB... " but w/o attribution of provenance nor any other > > > >> description of what it contains. Only when this is pointed-out in > the > > > issue > > > >> do we get a short listing of supposed JIRAs included with no > > > justification > > > >> of what is covered, what is included/excluded, and what machinations > > > were > > > >> done to make it fit branch-1 (Yet you offline were given this > info?). > > > >> > > > >> Such poor practice only makes me more intent on my objection. > > > >> > > > >> > > > >> > > > >>> Now, to your point, I agree that someone familiar with MOB code > > should > > > do > > > >>> the backport but the question is, is that dev available to do it > now? > > > The > > > >>> next best thing is to get Ted Yu's patch reviewed by someone > familiar > > > with > > > >>> the feature. I really hope that we can get cycles from the original > > MOB > > > >>> devs on this. > > > >> > > > >> > > > >> MOB is unsupported? If so, lets for sure not backport it. > > > >> > > > >> > > > >> Agree that we shouldn't be adding flaky tests. The question is if > the > > > >>> failures on master to do with MOB are really MOB related or > something > > > else > > > >>> (for argument's sake, let's say, procv2).. > > > >> Sounds like we need to spend a bit of time digging in on the flakey > > > tests > > > >> then. Branch-1 is pretty stable now after a bunch of expended effort > > by > > > a > > > >> few folks. Would be a pity taking a step back. > > > >> > > > >> St.Ack > > > >> > > > >> > > > >> > > > >>> On the point about the DISCUSS thread, yeah, it's fine to do it if > > > folks > > > >>> feel it's the right way to go. > > > >>> ________________________________________ > > > >>> From: [email protected] <[email protected]> on behalf of > Stack < > > > >>> [email protected]> > > > >>> Sent: Wednesday, March 02, 2016 10:55 AM > > > >>> To: HBase Dev List > > > >>> Subject: Re: MOB in branch-1? (Re: [RESULT][VOTE] Merge branch > > > hbase-11339 > > > >>> HBase MOB to trunk) > > > >>> > > > >>> (Doing another resend...) > > > >>> > > > >>> I have objection to you, Ted Yu, doing it. MOB has spread all about > > > master > > > >>> branch. Backport should be done by someone who knows MOB. > > > >>> > > > >>> Higher up in this thread, Sean asks: "Can we get a [DISCUSS] > flagged > > > thread > > > >>> to see what, if anything, folks > > > >>> would like to see gate inclusion in branch-1?" Shouldn't we do > this > > > before > > > >>> we 'create a backport...'. > > > >>> > > > >>> For me, there should be no new flakies in branch-1. Branch-1 builds > > > are a > > > >>> hard-won stable(-ish). Looking at master build, MOB seems quiet > > lately > > > but > > > >>> going by HBASE-15012, our flakies umbrella issue, I see notes that > > > >>> TestMobExportSnapshot has failed a few times (that is probably > > because > > > the > > > >>> test it derives from is flakey) and > > > TestMobRestoreFlushSnapshotFromClient > > > >>> gets a mention. > > > >>> > > > >>> St.Ack > > > >>> > > > >>> > > > >>> > > > >>>> On Tue, Mar 1, 2016 at 5:27 PM, Stack <[email protected]> wrote: > > > >>>> > > > >>>> (Doing a resend of below... it doesn't seem to have gone through) > > > >>>> > > > >>>>> On Mon, Feb 29, 2016 at 8:20 PM, Ted Yu <[email protected]> > > wrote: > > > >>>>> > > > >>>>> If there is no objection, I will create a backport JIRA tomorrow > > and > > > >>>>> attach patch. > > > >>>> I have objection to you doing it. MOB has spread all about master > > > branch. > > > >>>> Backport should be done by someone who knows MOB. > > > >>>> > > > >>>> Higher up in this thread, Sean asks: "Can we get a [DISCUSS] > flagged > > > >>>> thread to see what, if anything, folks > > > >>>> would like to see gate inclusion in branch-1?" Shouldn't we do > this > > > >>>> before we 'create a backport...'. > > > >>>> > > > >>>> For me, there should be no new flakies in branch-1. Branch-1 > builds > > > are a > > > >>>> hard-won stable(-ish). Looking at master build, MOB seems quiet > > lately > > > >>> but > > > >>>> going by HBASE-15012, our flakies umbrella issue, I see notes that > > > >>>> TestMobExportSnapshot has failed a few times (that is probably > > because > > > >>> the > > > >>>> test it derives from is flakey) and > > > TestMobRestoreFlushSnapshotFromClient > > > >>>> gets a mention. > > > >>>> > > > >>>> St.Ack > > > >>>> > > > >>>> > > > >>>> > > > >>>>>> On Tue, Mar 1, 2016 at 9:32 AM, Stack <[email protected]> wrote: > > > >>>>>> > > > >>>>>> On Mon, Feb 29, 2016 at 8:20 PM, Ted Yu <[email protected]> > > > wrote: > > > >>>>>> > > > >>>>>> If there is no objection, I will create a backport JIRA tomorrow > > and > > > >>>>>> attach patch. > > > >>>>> I have objection to you doing it. MOB has spread all about master > > > >>> branch. > > > >>>>> Backport should be done by someone who knows MOB. > > > >>>>> > > > >>>>> Higher up in this thread, Sean asks: "Can we get a [DISCUSS] > > flagged > > > >>>>> thread to see what, if anything, folks > > > >>>>> would like to see gate inclusion in branch-1?" Shouldn't we do > > this > > > >>>>> before we 'create a backport...'. > > > >>>>> > > > >>>>> For me, there should be no new flakies in branch-1. Branch-1 > builds > > > are > > > >>> a > > > >>>>> hard-won stable(-ish). Looking at master build, MOB seems quiet > > > lately > > > >>> but > > > >>>>> going by HBASE-15012, our flakies umbrella issue, I see notes > that > > > >>>>> TestMobExportSnapshot has failed a few times (that is probably > > > because > > > >>> the > > > >>>>> test it derives from is flakey) and > > > >>> TestMobRestoreFlushSnapshotFromClient > > > >>>>> gets a mention. > > > >>>>> > > > >>>>> St.Ack > > > >>> > > > > > > > > > > > -- > > // Jonathan Hsieh (shay) > > // HBase Tech Lead, Software Engineer, Cloudera > > // [email protected] // @jmhsieh > > > > > > -- > // Jonathan Hsieh (shay) > // HBase Tech Lead, Software Engineer, Cloudera > // [email protected] // @jmhsieh >
