Sounds good to me, Jonathan. Although I suspect that at the end this will be close to what Ted Yu has in the backport patch but this is a more structured / reliable way of doing it. When you say this - "backport the patches that came after the merge to trunk....", is there a reliable way to identify all those. In the backport jira, there is a mention of some additional jiras after the original merge - https://goo.gl/6stN7Y - we need to make sure if these are the only ones relevant. ________________________________________ From: Jonathan Hsieh <j...@cloudera.com> Sent: Wednesday, March 02, 2016 2:03 PM To: dev@hbase.apache.org Subject: Re: MOB in branch-1? (Re: [RESULT][VOTE] Merge branch hbase-11339 HBase MOB to trunk)
I'll offer what I think would be another reasonable and likely less destabilizing approach: essentially treat it as a branch. We could start from the existing hbase-11339 branch, call it hbase-11339-branch-1 and do the work to merge it into branch-1 (instead of trunk). To merge into branch-1 it would require the 3 +1's just like any other branch merge. Mob initially started off as a mega patch and was broken down and committed in reviewable pieces to branch hbase-11339. Most of this was actually when 1.0.0-SNAPSHOT was trunk so many of the older commits may actually jive better with branch-1. Towards the end we did periodic merge commits with trunk as we shook out issues integration testing found, and as branch merge reviews uncovered other short comings. These merge patches actually showed how we modified the mob code to account for the later changes in trunk. In this particular case we coud take hbase-11339-branch-1, backport the patches that came after the merge to trunk with the normal single +1 review process on the branch, and then propose a branch merge with branch-1. When all pieces are ready and we had stable unit tests (admittedly that was a one shortcoming when I sheparded hbase-11339) we do 3 +1s to merge to branch-1 (and possibly branch-1.3). This has the nice benefit compared to andrew's suggesting of not potentially blocking branch-1 or having a half complete mob feature in branch-1 if hbase-11339-branch-1 isn't ready. Jon. On Wed, Mar 2, 2016 at 1:31 PM, Andrew Purtell <andrew.purt...@gmail.com> wrote: > On the question of change provenance, if I can make a suggestion: How I > would approach the backport of a feature developed in trunk over multiple > commits and JIRAs is I would look through git history, find each related > commit, and apply each commit in sequence to branch-1, fixing things up as > needed each time. Then the resulting working branch can be pushed up to aid > review of the aggregate result by the MOB developers and the larger > community. Ted, if you approached the backport in this way, then pushing up > your working branch will aid review. > > > On Mar 2, 2016, at 1:17 PM, Andrew Purtell <andrew.purt...@gmail.com> > wrote: > > > > I concur with Stack's concerns with a developer not involved in building > the MOB feature attempting a backport in general, and furthermore without > detailed provenance of all of the incorporated changes. I'm also concerned > about how well MOB works in real production given people use branch-1 in > production and not trunk. > > > > As for Ted's issue, I'm not going to let him commit it (I will veto) > without clear proof that it works and doesn't hurt perf of non MOB cases by > way of reproducible benchmarks - benchmarks I can personally reproduce > afterward (I'm not volunteering to do Ted's necessary legwork) - done with > branch-1 with the proposed patch applied. We are not there yet though, > barely, into review, but this will be coming up very soon. > > > >>> On Mar 2, 2016, at 1:03 PM, Stack <st...@duboce.net> wrote: > >>> > >>> On Wed, Mar 2, 2016 at 11:26 AM, Devaraj Das <d...@hortonworks.com> > wrote: > >>> > >>> Stack, as things stand, Ted Yu has a patch that is a backport of MOB. > He > >>> told me offline he has taken a look at the jiras that went in in the > master > >>> that is to do with MOB, and got them in the backport. > >> > >> > >> > >> On the issue, a near 800k blob is dumped which is described as "...a > >> backport of MOB... " but w/o attribution of provenance nor any other > >> description of what it contains. Only when this is pointed-out in the > issue > >> do we get a short listing of supposed JIRAs included with no > justification > >> of what is covered, what is included/excluded, and what machinations > were > >> done to make it fit branch-1 (Yet you offline were given this info?). > >> > >> Such poor practice only makes me more intent on my objection. > >> > >> > >> > >>> Now, to your point, I agree that someone familiar with MOB code should > do > >>> the backport but the question is, is that dev available to do it now? > The > >>> next best thing is to get Ted Yu's patch reviewed by someone familiar > with > >>> the feature. I really hope that we can get cycles from the original MOB > >>> devs on this. > >> > >> > >> MOB is unsupported? If so, lets for sure not backport it. > >> > >> > >> Agree that we shouldn't be adding flaky tests. The question is if the > >>> failures on master to do with MOB are really MOB related or something > else > >>> (for argument's sake, let's say, procv2).. > >> Sounds like we need to spend a bit of time digging in on the flakey > tests > >> then. Branch-1 is pretty stable now after a bunch of expended effort by > a > >> few folks. Would be a pity taking a step back. > >> > >> St.Ack > >> > >> > >> > >>> On the point about the DISCUSS thread, yeah, it's fine to do it if > folks > >>> feel it's the right way to go. > >>> ________________________________________ > >>> From: saint....@gmail.com <saint....@gmail.com> on behalf of Stack < > >>> st...@duboce.net> > >>> Sent: Wednesday, March 02, 2016 10:55 AM > >>> To: HBase Dev List > >>> Subject: Re: MOB in branch-1? (Re: [RESULT][VOTE] Merge branch > hbase-11339 > >>> HBase MOB to trunk) > >>> > >>> (Doing another resend...) > >>> > >>> I have objection to you, Ted Yu, doing it. MOB has spread all about > master > >>> branch. Backport should be done by someone who knows MOB. > >>> > >>> Higher up in this thread, Sean asks: "Can we get a [DISCUSS] flagged > thread > >>> to see what, if anything, folks > >>> would like to see gate inclusion in branch-1?" Shouldn't we do this > before > >>> we 'create a backport...'. > >>> > >>> For me, there should be no new flakies in branch-1. Branch-1 builds > are a > >>> hard-won stable(-ish). Looking at master build, MOB seems quiet lately > but > >>> going by HBASE-15012, our flakies umbrella issue, I see notes that > >>> TestMobExportSnapshot has failed a few times (that is probably because > the > >>> test it derives from is flakey) and > TestMobRestoreFlushSnapshotFromClient > >>> gets a mention. > >>> > >>> St.Ack > >>> > >>> > >>> > >>>> On Tue, Mar 1, 2016 at 5:27 PM, Stack <st...@duboce.net> wrote: > >>>> > >>>> (Doing a resend of below... it doesn't seem to have gone through) > >>>> > >>>>> On Mon, Feb 29, 2016 at 8:20 PM, Ted Yu <yuzhih...@gmail.com> wrote: > >>>>> > >>>>> If there is no objection, I will create a backport JIRA tomorrow and > >>>>> attach patch. > >>>> I have objection to you doing it. MOB has spread all about master > branch. > >>>> Backport should be done by someone who knows MOB. > >>>> > >>>> Higher up in this thread, Sean asks: "Can we get a [DISCUSS] flagged > >>>> thread to see what, if anything, folks > >>>> would like to see gate inclusion in branch-1?" Shouldn't we do this > >>>> before we 'create a backport...'. > >>>> > >>>> For me, there should be no new flakies in branch-1. Branch-1 builds > are a > >>>> hard-won stable(-ish). Looking at master build, MOB seems quiet lately > >>> but > >>>> going by HBASE-15012, our flakies umbrella issue, I see notes that > >>>> TestMobExportSnapshot has failed a few times (that is probably because > >>> the > >>>> test it derives from is flakey) and > TestMobRestoreFlushSnapshotFromClient > >>>> gets a mention. > >>>> > >>>> St.Ack > >>>> > >>>> > >>>> > >>>>>> On Tue, Mar 1, 2016 at 9:32 AM, Stack <st...@duboce.net> wrote: > >>>>>> > >>>>>> On Mon, Feb 29, 2016 at 8:20 PM, Ted Yu <yuzhih...@gmail.com> > wrote: > >>>>>> > >>>>>> If there is no objection, I will create a backport JIRA tomorrow and > >>>>>> attach patch. > >>>>> I have objection to you doing it. MOB has spread all about master > >>> branch. > >>>>> Backport should be done by someone who knows MOB. > >>>>> > >>>>> Higher up in this thread, Sean asks: "Can we get a [DISCUSS] flagged > >>>>> thread to see what, if anything, folks > >>>>> would like to see gate inclusion in branch-1?" Shouldn't we do this > >>>>> before we 'create a backport...'. > >>>>> > >>>>> For me, there should be no new flakies in branch-1. Branch-1 builds > are > >>> a > >>>>> hard-won stable(-ish). Looking at master build, MOB seems quiet > lately > >>> but > >>>>> going by HBASE-15012, our flakies umbrella issue, I see notes that > >>>>> TestMobExportSnapshot has failed a few times (that is probably > because > >>> the > >>>>> test it derives from is flakey) and > >>> TestMobRestoreFlushSnapshotFromClient > >>>>> gets a mention. > >>>>> > >>>>> St.Ack > >>> > -- // Jonathan Hsieh (shay) // HBase Tech Lead, Software Engineer, Cloudera // j...@cloudera.com // @jmhsieh
