I just don't see a why we would back port. We're going to release a 2.0 when things are ready. It will be a major feature release. MOB is a major feature. Without compelling reason backporting to branch-1 seems like an end run around what sem ver is supposed to mean (not the api guarantees, the actual meaning).
Backporting major features is a bad habit that the Hadoop community seems to have. We shouldn't follow their lead. On Thu, Mar 3, 2016 at 1:01 PM, Stack <[email protected]> wrote: > On Thu, Mar 3, 2016 at 9:02 AM, Ted Yu <[email protected]> wrote: > > > Hi, > > As requested by Sean Busbey, I am starting a new thread w.r.t. > backporting > > MOB feature to branch-1. > > > This is to solicit discussion on the criteria for including MOB feature > > backport in branch-1. > > > > I can think of the following: > > 1. whether there is customer interest > > > > There is. > > See Jonathan's response here: > http://search-hadoop.com/m/YGbbDSqxD1PYXK62 > > > > > You should probably mention that a note requesting if there was interest > got no response here on the public lists. This would seem to imply no > interest by the community? > > Jon's note is pregnant but opaque on the details of these 'supported' > deploys. He is in a bit of an awkward spot unable to share detail on > someone else's deploy. > > Would be good to see more interest than Jon's note as evidence of interest > I'd say. > > You know of any? Even if they could be described in outline and hopefully > more than one instance and that MOB makes sense for this deploy. And they > need it now in a branch-1? > > Which version of hbase are we talking of backporting too? 1.3? 1.4? > > > > > 2. whether unit test stability can be maintained in branch-1 > > > > Inclusion of the backport should keep unit tests (both existing and new) > in > > branch-1 green. > > Preliminary test runs showed that MOB / snapshot related tests > consistently > > pass. > > > > > > > https://issues.apache.org/jira/browse/HBASE-15370?focusedCommentId=15176094&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15176094 > > > Hopefully we are aiming for more than just 'test stability'. Some large > community users have placed big bets on branch-1 being stable at scale; > this is the stability we should be precious of -- not just 'test > stability'. > > Besides, I see failures in your listing. MOB is pervasive. Why is it not > responsible for the mentioned test failures? (And "... passes on my local > machine... " doesn't cut it; builds.apache.org is our public CI where > community comes together on state of branch and master, not some devs > laptop). > > Branch-1 builds have been generally stable after large investment fixing, > tuning and pruning tests. Master branch had been rendered stable but seems > to be rotting again though it is the most important of our builds given it > can catch the bad stuff before the commits make it in. During the campaign > to stabilize Master, MOB tests failed often. I see MOB failures in master > patch build from time to time still (I've been lax of late but our flakies > list contains a few mentionds, see HBASE-15012). They go unaddressed > (though, to be fair, Jingcheng, the original author, addressed a few > failures early on when asked). Just this morning I note: > > https://builds.apache.org/view/H-L/view/HBase/job/PreCommit-HBASE-Build/818/testReport/junit/org.apache.hadoop.hbase.mob.mapreduce/TestMobSweepMapper/org_apache_hadoop_hbase_mob_mapreduce_TestMobSweepMapper/ > > > (Suggestion: MOB seems to be better in master branch of late -- Matteo and > Heng work I believe (or just my disabling of broke tests) -- so a review > and report of master and patch builds looking for MOB failures and fixing > any found would help address the above concern. Another way to build > confidence in a patched branch-1 would be doing the branch suggested up in > the backport issue and then running builds on apache for a period. Or > better, copy what was done by Jon et al. to build confidence; run > long-running ITBLLs on a cluster w/ MOB set with obnoxious configs and post > evidence it all holds up). > > > > Comments are welcome > > > > <https://issues.apache.org/jira/browse/HBASE-15370#> > > > > > Is MOB an 'owned' feature. The MOB original author is mostly absent (for > instance, no participation in these conversations and no general follow-up > on test failures). As has been said before many times, features need to be > owned. Writing the code is just one part of owning a feature. Features that > are not owned become a burden on the community to maintain. We have enough > of this latter type of feature already. > > (Jingcheng did show up in the last day though with HBASE-15381"Implement a > distributed MOB compaction by procedure" which looks like a pretty > important issue and begs the question, is MOB finished? And if not, > shouldn't we wait till its done before we backport?) > > Finally, MOB backport should be done by someone who is familiar with MOB. I > see no evidence of your expertise in MOB other than an odd review nor even > that you've run it in other than a unit test mode. Not to mention that the > way you go about the backport, dumping an 800k blob of unattributed code > into the issue for review in HBASE-15370, does not bode well for our > continued stability. > > St.Ack >
