For delete case,
Digging further into slider HDFS data in history directory via hdfs dfs
-cat
/user/xxx/.slider/cluster/ms999/history/rolehistory-0000015213700161.json;
it seems history keeps all roles that are active or inactive
{"entry":{"org.apache.slider.server.avro.RoleHistoryHeader":{"version":1,"saved":1452025053537,"savedx":"15213700161","savedate":"5
Jan 2016 20:17:33 GMT","roles":3}}}
{"entry":{"org.apache.slider.server.avro.RoleHistoryMapping":{"rolemap":{"slider-appmaster":0,"user1":1,"user2":2}}}}
{"entry":{"org.apache.slider.server.avro.NodeEntryRecord":{"host":"host1","role":1,"active":false,"last_used":1452025053537}}}
{"entry":{"org.apache.slider.server.avro.NodeEntryRecord":{"host":"host2","role":2,"active":true,"last_used":1452024753905}}}
{"entry":{"org.apache.slider.server.avro.RoleHistoryFooter":{"count":2}}}
Noted role active is false, as done by setting yarn.component.instances for
user1 to 0 ( I am sure flex 0 would do same ) to achive stopping component.
I understand this history will be required if someone did a flex 0 and then
need to do flex N (N > 0) later. However, in the use cases where this
component will never be used again; there should be a option to "delete"
the component from all configs permanently ...
Thanks,
Manoj
On Tue, Jan 5, 2016 at 9:52 AM, Manoj Samel <[email protected]>
wrote:
> Following is the delete / add use case for slider based long running
> service. The service has instances (a.k.a. components) provided per user.
>
> 1. When a new user is added, a new component is started for that user. The
> idea is not to bring the existing app / components down for existing users.
> Say first user "user1" joins and gets component named "user1". When "user2"
> joins, I add "user2" component to metainfo/appconfig/resource, redeploy the
> package and do a "slider upgrade". This keeps "user1" component running
> (don't want to bring down user1) and brings user2 component up.
>
> 2. When the user is decommissioned, its component should be decommissioned
> (permanently, if possible) without impacting other running components. Flex
> zero certainly works but as you pointed, it still remembers the component;
> which we may not want. I tried to do a "upgrade" with that component
> removed from all the configs - the AM fails after trying to "locate" the
> component I wanted to delete.
>
>
> 2016-01-05 02:31:35,802 [main] ERROR main.ServiceLauncher - Exception: Cannot
> find role for role ID 2
> java.lang.RuntimeException: Cannot find role for role ID 2
>
> at
> org.apache.slider.server.appmaster.state.AppState.lookupRoleStatus(AppState.java:917)
>
> at
> org.apache.slider.server.appmaster.state.AppState.addRestartedContainer(AppState.java:2242)
>
> at
> org.apache.slider.server.appmaster.state.AppState.rebuildModelFromRestart(AppState.java:2218)
>
> at
> org.apache.slider.server.appmaster.state.AppState.buildInstance(AppState.java:600)
>
> at
> org.apache.slider.server.appmaster.SliderAppMaster.createAndRunCluster(SliderAppMaster.java:849)
>
> at
> org.apache.slider.server.appmaster.SliderAppMaster.runService(SliderAppMaster.java:584)
>
> at
> org.apache.slider.core.main.ServiceLauncher.launchService(ServiceLauncher.java:188)
>
> at
> org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(ServiceLauncher.java:475)
>
> at
> org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(ServiceLauncher.java:403)
>
> at
> org.apache.slider.core.main.ServiceLauncher.serviceMain(ServiceLauncher.java:630)
>
> at
> org.apache.slider.server.appmaster.SliderAppMaster.main(SliderAppMaster.java:2297)
> 2016-01-05 02:31:35,807 [main] INFO util.ExitUtil - Exiting with status 56
>
> I may be able to live with flex for now but it would certainly be a good
> feature to be able to remove components permanently for use cases like
> above; where many users will come and go. If you think this is a valid
> feature ask, I can file a Jira
>
> Thanks,
>
> Manoj
>
> On Tue, Jan 5, 2016 at 2:42 AM, Steve Loughran <[email protected]>
> wrote:
>
>>
>> > On 5 Jan 2016, at 02:16, Manoj Samel <[email protected]> wrote:
>> >
>> > Hi,
>> >
>> > Use case needs adding and removing components dynamically.
>> >
>> > What is the proper way to remove a component ? It seems it can be done
>> > either by doing a "slider flex" with component count set to 0 or by
>> doing a
>> > "slider upgrade" where the component is deleted from metainfo.xml,
>> > appconfig.json and resources.json.
>> >
>> > Having trouble using "slider upgrade" to delete a component ... the
>> > documentation on upgrade is not clean on what it can / can't be used
>> for ...
>>
>>
>> Upgrade is actually for doing an update of the code of running code: it
>> upgrades each component one-by-one, then restarts the AM. Its goal is to
>> keep an app running while you update it.
>>
>> For setting a component count to zero, just use slider flex. Flexing down
>> should take seconds, and slider remembers where the component was. If your
>> component has any placement policy, then if you next flex up it will use
>> the most recently used locations for the initial requests
>>
>> -steve
>>
>
>
