Yes please file a jira. Let’s discuss it there further. On 1/18/16, 3:57 PM, "Manoj Samel" <manojsamelt...@gmail.com> wrote:
>Point take. > >For now, we have chosen to go with single app. And exactly to your point - >we don't want the AM to restart when we know that there is no change to be >picked up so restarting in this case has more downside (AM issue etc.) - >hence the feature ask. > >Any pros/conn on making this possible ? Should I file a feature jira ? > >Thanks > >On Mon, Jan 18, 2016 at 3:05 PM, Gour Saha <gs...@hortonworks.com> wrote: > >> Inline. >> >> On 1/14/16, 4:52 PM, "Manoj Samel" <manojsamelt...@gmail.com> wrote: >> >> >Hi Gour, >> > >> >The use case is described in >> >https://issues.apache.org/jira/browse/SLIDER-1046 >> > >> >Regarding your latest proposals - >> > >> >1. Can a "slider update" be used - As I recall, slider update is only >>when >> >application is stopped. We don't stop the application when adding new >>user >> >> [Gour] good point. I forgot that. >> > >> >2. Can a new slider app be used for each user - Hummm .. this is >> >interesting one ;^). After some thoughts, I see couple of issues >> > >> >Issue 1 - With large number of users, we have equally large number of >>AMs. >> >Even if AM footprint is small, it could grow in future and # of AM >>will be >> >a issue in any case. >> > >> >Issue 2 - Large number of applications will become issue. E.g. when >>doing >> >application upgrade, instead of doing one AM upgrade followed by >>rolling >> >upgrade of users, it will be larger number of AM upgrades. I think >>this is >> >less manageable. Also other operations like shutting down the app (in >> >event >> >of cluster downtime etc.) will be getting cumbersome >> > >> >Single app with different components per user seems still the best >>choice >> >... >> >> [Gour] Makes sense, but keep in mind that a single app is also subject >>to >> a single point of failure for all users (say in the unforeseen case of >>an >> issue during Slider AM or application upgrade). Note, multiple >> applications upgrade will be performed by a script or orchestrator, just >> like you would have to do to the equal number of components for a single >> app as well. >> > >> >Thanks, >> > >> >Manoj >> > >> >On Wed, Jan 13, 2016 at 2:08 PM, Gour Saha <gs...@hortonworks.com> >>wrote: >> > >> >> Can you look into ³slider update² (not upgrade) and then subsequently >> >> followed by appropriate ³slider flex² to bring up components for new >> >>users? >> >> >> >> It might be helpful if you can provide a brief explanation of your >> >> usecase. Just trying to see if mapping a new user to a new slider app >> >> (with a single component definition) makes more sense, compared to >>your >> >> current approach. The overhead of a Slider AM is very low. >> >> >> >> -Gour >> >> >> >> On 1/13/16, 10:35 AM, "Manoj Samel" <manojsamelt...@gmail.com> wrote: >> >> >> >> >Flex isn't option for couple of reasons in above use case >> >> > >> >> >1. For each new user, different parameters need to be passed when >> >> >starting. >> >> >New component allows control over component name and parameters >> >> >2. More importantly, when user is 'decomissioned', the component for >> >>that >> >> >user should be stopped for good. This is achieved at present by >>naming >> >> >each >> >> >component after respective user and stopping that when user is gone. >> >> > >> >> >Thanks, >> >> > >> >> >Manoj >> >> > >> >> >On Wed, Jan 13, 2016 at 10:11 AM, Steve Loughran >> >><ste...@hortonworks.com> >> >> >wrote: >> >> > >> >> >> you should just be flexing here >> >> >> >> >> >> > On 13 Jan 2016, at 09:19, Manoj Samel <manojsamelt...@gmail.com> >> >> >>wrote: >> >> >> > >> >> >> > Hi, >> >> >> > >> >> >> > In recent thread about existing components taking long time to >> >> >>heartbeat >> >> >> > after upgrade; it was mentioned that "slider AM restarts itself >>to >> >> >>pick >> >> >> up >> >> >> > Yarn parameters" >> >> >> > >> >> >> > If the upgrade simply consist of starting a new component, is >>the >> >>AM >> >> >> > restart still needed ? I.e. if the hadoop cluster configuration >>has >> >> >>not >> >> >> > changed and the Yarn parameters in the slider applications have >>not >> >> >> changed >> >> >> > for any components and upgrade is being used just to start a new >> >> >> component; >> >> >> > is AM restart still needed ? >> >> >> > >> >> >> > The use case is for service running large number of components. >> >>When a >> >> >> new >> >> >> > user is enabled for service, a new component is started for that >> >>user. >> >> >> > However, no changes are done for hadoop cluster or in the slider >> >> >> > application configuration for any existing components. When >>large >> >> >>number >> >> >> of >> >> >> > users (i.e. components) are already running, AM restart means >>these >> >> >> > existing components have to be informed of new AM and vice >>versa. >> >> >> > >> >> >> > In such cases, if AM restart is not needed, can it be avoided >>e.g. >> >>by >> >> >> > having a new option in Upgrade command ? >> >> >> > >> >> >> > Thanks, >> >> >> > >> >> >> > Manoj >> >> >> >> >> >> >> >> >> >> >> >>
