[ https://issues.apache.org/jira/browse/YARN-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15434851#comment-15434851 ]
Arun Suresh commented on YARN-1503: ----------------------------------- Thanks for driving this [~jianhe] I was wondering how this would tie into the NM Container state machine. Currently, a Container is in the LOCALIZING state while resources are being localized and then it starts. What state would the container be in now if re-Localization is allowed asynchronously ? What happens if localization failed for that specific resource ? Should the Container continue running ? Also looks like this overlaps a bit with what was being attempted in YARN-4876. In that JIRA, we do a couple of things: # it explicitly exposes an INIT state where the Container is LOCALIZED but not started yet. # it allows running containers to be re-initialized (and re-started) with new resources. The current prototype stops the container process, re-initializes and the re-starts it. It would be pretty straight forward to have a policy to not stop the container (keep the process running). Since we have explicit hooks in the container life-cycle, it would be possible to version the resources and provide support for roll-back etc. > Continuous resource-localization for YARN containers > ---------------------------------------------------- > > Key: YARN-1503 > URL: https://issues.apache.org/jira/browse/YARN-1503 > Project: Hadoop YARN > Issue Type: Improvement > Reporter: Siddharth Seth > Assignee: Jian He > Attachments: Continuous-resource-localization.pdf > > > We have a use case, where additional resources (jars, libraries etc) need to > be made available to an already running container. Ideally, we'd like this to > be done via YARN (instead of having potentially multiple containers per node > download resources on their own). > The goal is to enable NodeManagers to localize resources while container is > running. Today, resource-localization is always the first step before > starting a container. It will be useful if YARN can localize the resources > continuously even while container is running. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org