Thanks a bunch Nathan, for clearly letting us know the Yahoo! team's perspective.
We are getting started on rolling upgrades from YARN side (Sid opened YARN-666) and I hear HDFS side is too. We definitely need compatibility and testing kits. Have to get started on this. Work-preserving restart on YARN side - we plan to scope down next. Thanks, +Vinod On May 16, 2013, at 11:28 AM, Nathan Roberts wrote: > (initially respond on general@, sorry about that. copied here) > > +1 (non-binding) > > From my perspective: > > * The key feature that will drive me to adopt 2.x is Rolling Upgrades > * In order to get to rolling upgrades, we need a compatibility story that > is significantly better than we have today > ** We need a comprehensive definition of what compatibility really means > ** We need better testing in place to verify we're not breaking > compatibility > ** We need better definition and testing of what rolling upgrades really > means. Rolling between bug-fix releases Required, Rolling between minor > releases Required, Rolling between major releases Desired. > ** We need work-preserving restart on the YARN side. Restarting jobs > isn't sufficient. > ** ... > * Given that Rolling upgrades aren't there yet, and there is still work to > be done to solidify the compatibility story, I'm ok with the feature > window remaining open until these are in place, especially given the fact > that the proposed features are likely to have non-zero impact on > compatibility/rolling_upgrades. > * I'd certainly like a release with rolling upgrades as soon as possible, > so I feel like the feature window needs to ramp down very quickly. > Something like 2.0.5-beta in May with the current list of proposed > features, then 2.0.6-beta in late summer with full rolling upgrade support > and a solid compatibility story, would seem like a reasonable timeline. > Once we have a beta release with rolling upgrades, I can look at pushing > 2.x to some of our larger clusters. > > Nathan Roberts > nrobe...@yahoo-inc.com > > > > On 5/15/13 1:06 PM, "Vinod Kumar Vavilapalli" <vino...@hortonworks.com> > wrote: > >> >> Seems like you forgot to bcc. Forwarding this to general. >> >> Thanks, >> +Vinod >> On May 15, 2013, at 10:57 AM, Arun C Murthy wrote: >> >>> Folks, >>> >>> A considerable number of people have expressed confusion regarding the >>> recent vote on 2.0.5, beta status etc. given lack of specifics, the >>> voting itself (validity of the vote itself, whose votes are binding) etc. >>> >>> IMHO technical arguments (incompatibility b/w 2.0 & 2.1, current >>> stability of 3 features under debate etc.) have been lost in the >>> discussion in favor of non-technical (almost dramatic) nuances such as >>> "seizing the moment". There is now dangerous talk of tolerating >>> incompatibility b/w 2.0 and 2.1) - this is a red flag for me; >>> particularly when there are just 3 features being debated and active >>> committers and contributors are confident of and ready to stand by their >>> work. All patches, I believe, are ready to be merged in the the next few >>> days per discussions on jira. This will, clearly, not delay the other >>> API work which everyone agrees is crucial. As a result, I feel no >>> recourse but to restart a new vote - all attempts at calm, reasoned, >>> civil discussion based on technical arguments have come to naught - I >>> apologize for the thrash caused to everyone's attention. >>> >>> To get past all of this confusion, I'd like to present an alternate, >>> specific proposal for consideration. >>> >>> I propose we continue the original plan and make a 2.0.5-beta release >>> by May end with the following content: >>> # HDFS-347 >>> # HDFS Snapshots >>> # Windows support >>> # Necessary & final API/protocol changes such as: >>> * Final YARN API changes: YARN-386 >>> * MR Binary Compatibility: MAPREDUCE-5108 >>> * Final RPC cleanup: HADOOP-8990 >>> >>> People working on the above features have all expressed considerable >>> comfort with them and are ready to stand-by to help expedite any >>> necessary bug-fixes etc. to get to stabilization quickly. I'm confident >>> we can get this release out by end of May. This sets stage for a >>> hadoop-2.x GA release right after with some more testing - this means I >>> think I can quickly turn around and make bug-fix releases as necessary >>> right after 2.0.5-beta. >>> >>> I request that people consider helping out with this plan and sign up >>> to help push hadoop-2.x to stability as outlined above. I believe this >>> will help achieve our shared goals of quickly stabilizing hadoop-2 and >>> help ensure we can support it for forseeable future in a compatible >>> manner for the benefit of our users and downstream projects. >>> >>> Please vote, the vote will run the normal 7 days. Obviously, I'm +1. >>> >>> thanks, >>> Arun >>> >>> PS: To keep this discussion grounded in technical details I've moved >>> this to dev@ (bcc general@). >>> >> >
