[
https://issues.apache.org/jira/browse/YARN-1039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14041086#comment-14041086
]
Craig Welch commented on YARN-1039:
-----------------------------------
[~ste...@apache.org] wrt the need for a container level flag / a way for the
application master to launch long lived containers - definitely, but the idea
was for that to come as a later step - although that may be short-sighted, as
it may be better to come up with a common way to do this for the application
master container and the containers it later launches now instead of ending up
with unmatched approaches later...
This first step is to provide a way for the application master to be launched
in a long lived container (generally, an application master for a long lived
application will need to itself be launched in a long lived container - at
least, it needs to be possible to do so) - which is why there needs to be some
way to indicate the need for a long lived container during application
submission (necessary but not sufficient overall...)
[~zjshen] I was also wondering about using the tags, but after talking with
[~xgong] we are not thinking that is the way to go because tags don't seem to
be about changing behavior but only about freeform way to enable
search/display/etc.
After this discussion and some looking around it really seems that what we are
after is a way to communicate a quality of the needed container to the resource
manager both at application submission (for the application master container)
and also for later container launches by the master, kind of like the
ResourceProto, which is also already present in both cases for the same reason
(I suggested adding it there, actually, as something "necessary for the
container" but [~xgong] objected, thinking it is really specific to metric
qualities (cpu, memory...).
I'm going to take a look at adding something alongside /similar to the
ResourceProto to indicate constraints/requirements for the container, starting
with long lived, that can be common to application submission and when the
containers are started later by the application, not necessarily a long field
for bit manipulation but something which is also extensible
> Add parameter for YARN resource requests to indicate "long lived"
> -----------------------------------------------------------------
>
> Key: YARN-1039
> URL: https://issues.apache.org/jira/browse/YARN-1039
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
> Affects Versions: 3.0.0, 2.1.1-beta
> Reporter: Steve Loughran
> Assignee: Craig Welch
> Priority: Minor
> Attachments: YARN-1039.1.patch, YARN-1039.2.patch, YARN-1039.3.patch
>
>
> A container request could support a new parameter "long-lived". This could be
> used by a scheduler that would know not to host the service on a transient
> (cloud: spot priced) node.
> Schedulers could also decide whether or not to allocate multiple long-lived
> containers on the same node
--
This message was sent by Atlassian JIRA
(v6.2#6252)
