Inline. On 1/14/16, 4:52 PM, "Manoj Samel" <manojsamelt...@gmail.com> wrote:
>Hi Gour, > >The use case is described in >https://issues.apache.org/jira/browse/SLIDER-1046 > >Regarding your latest proposals - > >1. Can a "slider update" be used - As I recall, slider update is only when >application is stopped. We don't stop the application when adding new user [Gour] good point. I forgot that. > >2. Can a new slider app be used for each user - Hummm .. this is >interesting one ;^). After some thoughts, I see couple of issues > >Issue 1 - With large number of users, we have equally large number of AMs. >Even if AM footprint is small, it could grow in future and # of AM will be >a issue in any case. > >Issue 2 - Large number of applications will become issue. E.g. when doing >application upgrade, instead of doing one AM upgrade followed by rolling >upgrade of users, it will be larger number of AM upgrades. I think this is >less manageable. Also other operations like shutting down the app (in >event >of cluster downtime etc.) will be getting cumbersome > >Single app with different components per user seems still the best choice >... [Gour] Makes sense, but keep in mind that a single app is also subject to a single point of failure for all users (say in the unforeseen case of an issue during Slider AM or application upgrade). Note, multiple applications upgrade will be performed by a script or orchestrator, just like you would have to do to the equal number of components for a single app as well. > >Thanks, > >Manoj > >On Wed, Jan 13, 2016 at 2:08 PM, Gour Saha <gs...@hortonworks.com> wrote: > >> Can you look into ³slider update² (not upgrade) and then subsequently >> followed by appropriate ³slider flex² to bring up components for new >>users? >> >> It might be helpful if you can provide a brief explanation of your >> usecase. Just trying to see if mapping a new user to a new slider app >> (with a single component definition) makes more sense, compared to your >> current approach. The overhead of a Slider AM is very low. >> >> -Gour >> >> On 1/13/16, 10:35 AM, "Manoj Samel" <manojsamelt...@gmail.com> wrote: >> >> >Flex isn't option for couple of reasons in above use case >> > >> >1. For each new user, different parameters need to be passed when >> >starting. >> >New component allows control over component name and parameters >> >2. More importantly, when user is 'decomissioned', the component for >>that >> >user should be stopped for good. This is achieved at present by naming >> >each >> >component after respective user and stopping that when user is gone. >> > >> >Thanks, >> > >> >Manoj >> > >> >On Wed, Jan 13, 2016 at 10:11 AM, Steve Loughran >><ste...@hortonworks.com> >> >wrote: >> > >> >> you should just be flexing here >> >> >> >> > On 13 Jan 2016, at 09:19, Manoj Samel <manojsamelt...@gmail.com> >> >>wrote: >> >> > >> >> > Hi, >> >> > >> >> > In recent thread about existing components taking long time to >> >>heartbeat >> >> > after upgrade; it was mentioned that "slider AM restarts itself to >> >>pick >> >> up >> >> > Yarn parameters" >> >> > >> >> > If the upgrade simply consist of starting a new component, is the >>AM >> >> > restart still needed ? I.e. if the hadoop cluster configuration has >> >>not >> >> > changed and the Yarn parameters in the slider applications have not >> >> changed >> >> > for any components and upgrade is being used just to start a new >> >> component; >> >> > is AM restart still needed ? >> >> > >> >> > The use case is for service running large number of components. >>When a >> >> new >> >> > user is enabled for service, a new component is started for that >>user. >> >> > However, no changes are done for hadoop cluster or in the slider >> >> > application configuration for any existing components. When large >> >>number >> >> of >> >> > users (i.e. components) are already running, AM restart means these >> >> > existing components have to be informed of new AM and vice versa. >> >> > >> >> > In such cases, if AM restart is not needed, can it be avoided e.g. >>by >> >> > having a new option in Upgrade command ? >> >> > >> >> > Thanks, >> >> > >> >> > Manoj >> >> >> >> >> >>
