+1. This should go a long way to help running services on YARN. -Varun
On 7/19/16, 12:53 AM, "Gour Saha" <gs...@hortonworks.com> wrote: >+1 forwarding from the discussion in Slider DL > >Note: On the Slider side, we plan to create a branch corresponding to this >YARN branch. In this, we intend to remove all the pieces which will move >to the YARN branch (Slider core, AM, client). We will retain the Agent and >app-packages which will then depend on the new hadoop-slider module (in >addition to the existing hadoop modules that it already depends on). This >will create a single view of Slider codebase, exactly as it stands today, >fully supporting the current app-packages. Slider can even release its >future versions from this new branch, once the hadoop-sldier module >reaches a matured state. It will be an easier path for existing Slider >users/app-owners to move to the future stable state of Slider completely >off of Hadoop YARN codebase. All that would be expected is to migrate the >app-packages to the agent-less version. Of course the Slider community >will do the migration for the current app-packages in the Slider codebase. > >-Gour > >On 7/14/16, 7:36 PM, "Vinod Kumar Vavilapalli" <vino...@apache.org> wrote: > >>Hi, Hadoop YARN community! >> >>(Cross-posting across Hadoop and Slider communities) >> >>I opened a JIRA a little while ago to pursue a native YARN framework for >>services: https://issues.apache.org/jira/browse/YARN-5079. >> >>It is part of a bigger effort that a bunch of us YARN community members >>are interested in making progress on: YARN-4692 - [Umbrella] Simplified >>and first-class support for services in YARN. >> >>The idea is that with our current attention on making services >>first–-class, it's time to take a fresh look at how we can make Apache >>Hadoop YARN support services well out of the box. I’ve been looking at >>various possibilities - ranging from a custom new framework room scratch >>to using one of the existing projects - and stopped at Apache Slider >>(http://slider.incubator.apache.org) given its association with some of >>the YARN community members (Steve Loughran, Devaraj Das, Arun C Murthy, >>myself etc.). >> >>Slider client & AM already handles a great deal of the functionality that >>we need. I posit that assimilating the client, ApplicationMaster etc of >>an existing framework like Apache Slider can serve our purpose really >>well. My early informal discussions about this with few Hadoop and Slider >>community members yielded generally favourable feedback. >> >>The Apache Slider incubator community also discussed this and expressed >>generally positive interest in YARN taking up Slider’s key pieces, you >>can see that discussion here: https://s.apache.org/0hoh. >> >>So in summary, we are looking to the following >> >>- Code >> — ‘Graduate' key pieces (Slider client, AM) of Apache Slider into >>Apache Hadoop for providing a native services experiences in YARN >> — Leave for now some of the pieces behind in Apache Slider - (a) >>Slider agent as we won’t need it, (b) Slider packages that need more >>deliberation in terms of where they will live in the long term. >> — Create a branch in YARN, copy this code over into a new module(s), >>and work towards completing a functioning app running on YARN. >> >>- Communities & releases >> — Good thing is that many of Apache Slider community members are >>already seasoned folk in the Apache Hadoop ecosystem projects. For those >>committers & PMC in Slider that are not yet Hadoop committers / PMC, >>without complicating things much, a proposed path forward is active >>participation in the branch (as branch committers?) and eventually in >>mainline YARN and thus go through a natural progression to committership >>/ PMC. Given that most of the members are stalwarts in the Apache >>communities, this should be a cinch IMO. >> — The work on this new code can start, and depending on its state, and >>assuming that the experiment succeeds, can be merged into trunk and later >>picked up in the next nearest & feasible Apache Hadoop release. >> — While the work on forked-over-code goes on till we have a >>functioning app, the current Apache Slider project continues to live, >>with supporting releases etc. At some point in the future, when things >>become clearer, more decisions can be taken on the (parts left behind in >>the) incubating project’s future. >> >>Thoughts? >> >>Thanks >>+Vinod > > >--------------------------------------------------------------------- >To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org >For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org >
