+1. Merging the core parts of Slider into YARN should help with faster development on service-oriented features.
On Tue, Jul 19, 2016 at 12:38 PM, Wangda Tan <wheele...@gmail.com> wrote: > +1, this can avoid lots efforts for end user to run services on YARN. > > On Tue, Jul 19, 2016 at 7:12 AM, Billie Rinaldi <billie.rina...@gmail.com> > wrote: > > > +1. I think making Slider the basis of a services framework in YARN will > > allow us to significantly improve YARN's support for services. > > > > On Thu, Jul 14, 2016 at 7:36 PM, Vinod Kumar Vavilapalli < > > vino...@apache.org > > > wrote: > > > > > Hi, Hadoop YARN community! > > > > > > (Cross-posting across Hadoop and Slider communities) > > > > > > I opened a JIRA a little while ago to pursue a native YARN framework > for > > > services: https://issues.apache.org/jira/browse/YARN-5079. > > > > > > It is part of a bigger effort that a bunch of us YARN community members > > > are interested in making progress on: YARN-4692 - [Umbrella] Simplified > > and > > > first-class support for services in YARN. > > > > > > The idea is that with our current attention on making services > > > first-class, it's time to take a fresh look at how we can make Apache > > > Hadoop YARN support services well out of the box. I’ve been looking at > > > various possibilities - ranging from a custom new framework room > scratch > > to > > > using one of the existing projects - and stopped at Apache Slider ( > > > http://slider.incubator.apache.org) given its association with some of > > > the YARN community members (Steve Loughran, Devaraj Das, Arun C Murthy, > > > myself etc.). > > > > > > Slider client & AM already handles a great deal of the functionality > that > > > we need. I posit that assimilating the client, ApplicationMaster etc of > > an > > > existing framework like Apache Slider can serve our purpose really > well. > > My > > > early informal discussions about this with few Hadoop and Slider > > community > > > members yielded generally favourable feedback. > > > > > > The Apache Slider incubator community also discussed this and expressed > > > generally positive interest in YARN taking up Slider’s key pieces, you > > can > > > see that discussion here: https://s.apache.org/0hoh. > > > > > > So in summary, we are looking to the following > > > > > > - Code > > > — ‘Graduate' key pieces (Slider client, AM) of Apache Slider into > > > Apache Hadoop for providing a native services experiences in YARN > > > — Leave for now some of the pieces behind in Apache Slider - (a) > > Slider > > > agent as we won’t need it, (b) Slider packages that need more > > deliberation > > > in terms of where they will live in the long term. > > > — Create a branch in YARN, copy this code over into a new module(s), > > > and work towards completing a functioning app running on YARN. > > > > > > - Communities & releases > > > — Good thing is that many of Apache Slider community members are > > > already seasoned folk in the Apache Hadoop ecosystem projects. For > those > > > committers & PMC in Slider that are not yet Hadoop committers / PMC, > > > without complicating things much, a proposed path forward is active > > > participation in the branch (as branch committers?) and eventually in > > > mainline YARN and thus go through a natural progression to > committership > > / > > > PMC. Given that most of the members are stalwarts in the Apache > > > communities, this should be a cinch IMO. > > > — The work on this new code can start, and depending on its state, > and > > > assuming that the experiment succeeds, can be merged into trunk and > later > > > picked up in the next nearest & feasible Apache Hadoop release. > > > — While the work on forked-over-code goes on till we have a > > functioning > > > app, the current Apache Slider project continues to live, with > supporting > > > releases etc. At some point in the future, when things become clearer, > > more > > > decisions can be taken on the (parts left behind in the) incubating > > > project’s future. > > > > > > Thoughts? > > > > > > Thanks > > > +Vinod > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org > > > For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org > > > > > > > > >
