+1, this can avoid lots efforts for end user to run services on YARN. On Tue, Jul 19, 2016 at 7:12 AM, Billie Rinaldi <billie.rina...@gmail.com> wrote:
> +1. I think making Slider the basis of a services framework in YARN will > allow us to significantly improve YARN's support for services. > > On Thu, Jul 14, 2016 at 7:36 PM, Vinod Kumar Vavilapalli < > vino...@apache.org > > wrote: > > > Hi, Hadoop YARN community! > > > > (Cross-posting across Hadoop and Slider communities) > > > > I opened a JIRA a little while ago to pursue a native YARN framework for > > services: https://issues.apache.org/jira/browse/YARN-5079. > > > > It is part of a bigger effort that a bunch of us YARN community members > > are interested in making progress on: YARN-4692 - [Umbrella] Simplified > and > > first-class support for services in YARN. > > > > The idea is that with our current attention on making services > > first-class, it's time to take a fresh look at how we can make Apache > > Hadoop YARN support services well out of the box. I’ve been looking at > > various possibilities - ranging from a custom new framework room scratch > to > > using one of the existing projects - and stopped at Apache Slider ( > > http://slider.incubator.apache.org) given its association with some of > > the YARN community members (Steve Loughran, Devaraj Das, Arun C Murthy, > > myself etc.). > > > > Slider client & AM already handles a great deal of the functionality that > > we need. I posit that assimilating the client, ApplicationMaster etc of > an > > existing framework like Apache Slider can serve our purpose really well. > My > > early informal discussions about this with few Hadoop and Slider > community > > members yielded generally favourable feedback. > > > > The Apache Slider incubator community also discussed this and expressed > > generally positive interest in YARN taking up Slider’s key pieces, you > can > > see that discussion here: https://s.apache.org/0hoh. > > > > So in summary, we are looking to the following > > > > - Code > > — ‘Graduate' key pieces (Slider client, AM) of Apache Slider into > > Apache Hadoop for providing a native services experiences in YARN > > — Leave for now some of the pieces behind in Apache Slider - (a) > Slider > > agent as we won’t need it, (b) Slider packages that need more > deliberation > > in terms of where they will live in the long term. > > — Create a branch in YARN, copy this code over into a new module(s), > > and work towards completing a functioning app running on YARN. > > > > - Communities & releases > > — Good thing is that many of Apache Slider community members are > > already seasoned folk in the Apache Hadoop ecosystem projects. For those > > committers & PMC in Slider that are not yet Hadoop committers / PMC, > > without complicating things much, a proposed path forward is active > > participation in the branch (as branch committers?) and eventually in > > mainline YARN and thus go through a natural progression to committership > / > > PMC. Given that most of the members are stalwarts in the Apache > > communities, this should be a cinch IMO. > > — The work on this new code can start, and depending on its state, and > > assuming that the experiment succeeds, can be merged into trunk and later > > picked up in the next nearest & feasible Apache Hadoop release. > > — While the work on forked-over-code goes on till we have a > functioning > > app, the current Apache Slider project continues to live, with supporting > > releases etc. At some point in the future, when things become clearer, > more > > decisions can be taken on the (parts left behind in the) incubating > > project’s future. > > > > Thoughts? > > > > Thanks > > +Vinod > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org > > For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org > > > > >
