Hi Gour, The use case is described in https://issues.apache.org/jira/browse/SLIDER-1046
Regarding your latest proposals - 1. Can a "slider update" be used - As I recall, slider update is only when application is stopped. We don't stop the application when adding new user 2. Can a new slider app be used for each user - Hummm .. this is interesting one ;^). After some thoughts, I see couple of issues Issue 1 - With large number of users, we have equally large number of AMs. Even if AM footprint is small, it could grow in future and # of AM will be a issue in any case. Issue 2 - Large number of applications will become issue. E.g. when doing application upgrade, instead of doing one AM upgrade followed by rolling upgrade of users, it will be larger number of AM upgrades. I think this is less manageable. Also other operations like shutting down the app (in event of cluster downtime etc.) will be getting cumbersome Single app with different components per user seems still the best choice ... Thanks, Manoj On Wed, Jan 13, 2016 at 2:08 PM, Gour Saha <gs...@hortonworks.com> wrote: > Can you look into ³slider update² (not upgrade) and then subsequently > followed by appropriate ³slider flex² to bring up components for new users? > > It might be helpful if you can provide a brief explanation of your > usecase. Just trying to see if mapping a new user to a new slider app > (with a single component definition) makes more sense, compared to your > current approach. The overhead of a Slider AM is very low. > > -Gour > > On 1/13/16, 10:35 AM, "Manoj Samel" <manojsamelt...@gmail.com> wrote: > > >Flex isn't option for couple of reasons in above use case > > > >1. For each new user, different parameters need to be passed when > >starting. > >New component allows control over component name and parameters > >2. More importantly, when user is 'decomissioned', the component for that > >user should be stopped for good. This is achieved at present by naming > >each > >component after respective user and stopping that when user is gone. > > > >Thanks, > > > >Manoj > > > >On Wed, Jan 13, 2016 at 10:11 AM, Steve Loughran <ste...@hortonworks.com> > >wrote: > > > >> you should just be flexing here > >> > >> > On 13 Jan 2016, at 09:19, Manoj Samel <manojsamelt...@gmail.com> > >>wrote: > >> > > >> > Hi, > >> > > >> > In recent thread about existing components taking long time to > >>heartbeat > >> > after upgrade; it was mentioned that "slider AM restarts itself to > >>pick > >> up > >> > Yarn parameters" > >> > > >> > If the upgrade simply consist of starting a new component, is the AM > >> > restart still needed ? I.e. if the hadoop cluster configuration has > >>not > >> > changed and the Yarn parameters in the slider applications have not > >> changed > >> > for any components and upgrade is being used just to start a new > >> component; > >> > is AM restart still needed ? > >> > > >> > The use case is for service running large number of components. When a > >> new > >> > user is enabled for service, a new component is started for that user. > >> > However, no changes are done for hadoop cluster or in the slider > >> > application configuration for any existing components. When large > >>number > >> of > >> > users (i.e. components) are already running, AM restart means these > >> > existing components have to be informed of new AM and vice versa. > >> > > >> > In such cases, if AM restart is not needed, can it be avoided e.g. by > >> > having a new option in Upgrade command ? > >> > > >> > Thanks, > >> > > >> > Manoj > >> > >> > >
