Point take. For now, we have chosen to go with single app. And exactly to your point - we don't want the AM to restart when we know that there is no change to be picked up so restarting in this case has more downside (AM issue etc.) - hence the feature ask.
Any pros/conn on making this possible ? Should I file a feature jira ? Thanks On Mon, Jan 18, 2016 at 3:05 PM, Gour Saha <gs...@hortonworks.com> wrote: > Inline. > > On 1/14/16, 4:52 PM, "Manoj Samel" <manojsamelt...@gmail.com> wrote: > > >Hi Gour, > > > >The use case is described in > >https://issues.apache.org/jira/browse/SLIDER-1046 > > > >Regarding your latest proposals - > > > >1. Can a "slider update" be used - As I recall, slider update is only when > >application is stopped. We don't stop the application when adding new user > > [Gour] good point. I forgot that. > > > >2. Can a new slider app be used for each user - Hummm .. this is > >interesting one ;^). After some thoughts, I see couple of issues > > > >Issue 1 - With large number of users, we have equally large number of AMs. > >Even if AM footprint is small, it could grow in future and # of AM will be > >a issue in any case. > > > >Issue 2 - Large number of applications will become issue. E.g. when doing > >application upgrade, instead of doing one AM upgrade followed by rolling > >upgrade of users, it will be larger number of AM upgrades. I think this is > >less manageable. Also other operations like shutting down the app (in > >event > >of cluster downtime etc.) will be getting cumbersome > > > >Single app with different components per user seems still the best choice > >... > > [Gour] Makes sense, but keep in mind that a single app is also subject to > a single point of failure for all users (say in the unforeseen case of an > issue during Slider AM or application upgrade). Note, multiple > applications upgrade will be performed by a script or orchestrator, just > like you would have to do to the equal number of components for a single > app as well. > > > >Thanks, > > > >Manoj > > > >On Wed, Jan 13, 2016 at 2:08 PM, Gour Saha <gs...@hortonworks.com> wrote: > > > >> Can you look into ³slider update² (not upgrade) and then subsequently > >> followed by appropriate ³slider flex² to bring up components for new > >>users? > >> > >> It might be helpful if you can provide a brief explanation of your > >> usecase. Just trying to see if mapping a new user to a new slider app > >> (with a single component definition) makes more sense, compared to your > >> current approach. The overhead of a Slider AM is very low. > >> > >> -Gour > >> > >> On 1/13/16, 10:35 AM, "Manoj Samel" <manojsamelt...@gmail.com> wrote: > >> > >> >Flex isn't option for couple of reasons in above use case > >> > > >> >1. For each new user, different parameters need to be passed when > >> >starting. > >> >New component allows control over component name and parameters > >> >2. More importantly, when user is 'decomissioned', the component for > >>that > >> >user should be stopped for good. This is achieved at present by naming > >> >each > >> >component after respective user and stopping that when user is gone. > >> > > >> >Thanks, > >> > > >> >Manoj > >> > > >> >On Wed, Jan 13, 2016 at 10:11 AM, Steve Loughran > >><ste...@hortonworks.com> > >> >wrote: > >> > > >> >> you should just be flexing here > >> >> > >> >> > On 13 Jan 2016, at 09:19, Manoj Samel <manojsamelt...@gmail.com> > >> >>wrote: > >> >> > > >> >> > Hi, > >> >> > > >> >> > In recent thread about existing components taking long time to > >> >>heartbeat > >> >> > after upgrade; it was mentioned that "slider AM restarts itself to > >> >>pick > >> >> up > >> >> > Yarn parameters" > >> >> > > >> >> > If the upgrade simply consist of starting a new component, is the > >>AM > >> >> > restart still needed ? I.e. if the hadoop cluster configuration has > >> >>not > >> >> > changed and the Yarn parameters in the slider applications have not > >> >> changed > >> >> > for any components and upgrade is being used just to start a new > >> >> component; > >> >> > is AM restart still needed ? > >> >> > > >> >> > The use case is for service running large number of components. > >>When a > >> >> new > >> >> > user is enabled for service, a new component is started for that > >>user. > >> >> > However, no changes are done for hadoop cluster or in the slider > >> >> > application configuration for any existing components. When large > >> >>number > >> >> of > >> >> > users (i.e. components) are already running, AM restart means these > >> >> > existing components have to be informed of new AM and vice versa. > >> >> > > >> >> > In such cases, if AM restart is not needed, can it be avoided e.g. > >>by > >> >> > having a new option in Upgrade command ? > >> >> > > >> >> > Thanks, > >> >> > > >> >> > Manoj > >> >> > >> >> > >> > >> > >
