On Thu, Mar 3, 2016 at 9:02 AM, Ted Yu <[email protected]> wrote: > Hi, > As requested by Sean Busbey, I am starting a new thread w.r.t. backporting > MOB feature to branch-1. > This is to solicit discussion on the criteria for including MOB feature > backport in branch-1. > > I can think of the following: > 1. whether there is customer interest > > There is. > See Jonathan's response here: http://search-hadoop.com/m/YGbbDSqxD1PYXK62 > > You should probably mention that a note requesting if there was interest got no response here on the public lists. This would seem to imply no interest by the community?
Jon's note is pregnant but opaque on the details of these 'supported' deploys. He is in a bit of an awkward spot unable to share detail on someone else's deploy. Would be good to see more interest than Jon's note as evidence of interest I'd say. You know of any? Even if they could be described in outline and hopefully more than one instance and that MOB makes sense for this deploy. And they need it now in a branch-1? Which version of hbase are we talking of backporting too? 1.3? 1.4? > 2. whether unit test stability can be maintained in branch-1 > > Inclusion of the backport should keep unit tests (both existing and new) in > branch-1 green. > Preliminary test runs showed that MOB / snapshot related tests consistently > pass. > > > https://issues.apache.org/jira/browse/HBASE-15370?focusedCommentId=15176094&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15176094 Hopefully we are aiming for more than just 'test stability'. Some large community users have placed big bets on branch-1 being stable at scale; this is the stability we should be precious of -- not just 'test stability'. Besides, I see failures in your listing. MOB is pervasive. Why is it not responsible for the mentioned test failures? (And "... passes on my local machine... " doesn't cut it; builds.apache.org is our public CI where community comes together on state of branch and master, not some devs laptop). Branch-1 builds have been generally stable after large investment fixing, tuning and pruning tests. Master branch had been rendered stable but seems to be rotting again though it is the most important of our builds given it can catch the bad stuff before the commits make it in. During the campaign to stabilize Master, MOB tests failed often. I see MOB failures in master patch build from time to time still (I've been lax of late but our flakies list contains a few mentionds, see HBASE-15012). They go unaddressed (though, to be fair, Jingcheng, the original author, addressed a few failures early on when asked). Just this morning I note: https://builds.apache.org/view/H-L/view/HBase/job/PreCommit-HBASE-Build/818/testReport/junit/org.apache.hadoop.hbase.mob.mapreduce/TestMobSweepMapper/org_apache_hadoop_hbase_mob_mapreduce_TestMobSweepMapper/ (Suggestion: MOB seems to be better in master branch of late -- Matteo and Heng work I believe (or just my disabling of broke tests) -- so a review and report of master and patch builds looking for MOB failures and fixing any found would help address the above concern. Another way to build confidence in a patched branch-1 would be doing the branch suggested up in the backport issue and then running builds on apache for a period. Or better, copy what was done by Jon et al. to build confidence; run long-running ITBLLs on a cluster w/ MOB set with obnoxious configs and post evidence it all holds up). > Comments are welcome > > <https://issues.apache.org/jira/browse/HBASE-15370#> > Is MOB an 'owned' feature. The MOB original author is mostly absent (for instance, no participation in these conversations and no general follow-up on test failures). As has been said before many times, features need to be owned. Writing the code is just one part of owning a feature. Features that are not owned become a burden on the community to maintain. We have enough of this latter type of feature already. (Jingcheng did show up in the last day though with HBASE-15381"Implement a distributed MOB compaction by procedure" which looks like a pretty important issue and begs the question, is MOB finished? And if not, shouldn't we wait till its done before we backport?) Finally, MOB backport should be done by someone who is familiar with MOB. I see no evidence of your expertise in MOB other than an odd review nor even that you've run it in other than a unit test mode. Not to mention that the way you go about the backport, dumping an 800k blob of unattributed code into the issue for review in HBASE-15370, does not bode well for our continued stability. St.Ack
