[
https://issues.apache.org/jira/browse/YARN-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15434227#comment-15434227
]
Jian He commented on YARN-1503:
-------------------------------
Jason, Karthik, thanks for the comments !
bq. Omit the localization info for NM heartbeats? The RM currently doesn't
care, so sending it is a waste.
Good point. I'll make sure it does not impact regular heartbeats.
>From implementation perspective, I'll work on the core change and the localize
>API first, and defer the localization-status part later when it is required,
>as we are also fleshing out how the caller can make use of this. I had also
>talked with [~sseth] previously, with the support of HADOOP-11552, we can even
>have a better 'async' notification of when the localization is done, rather
>than current polling mechanism.
bq. One other approach we had considered was ability to list all resources in
a file
Converting to a file will introduce other string parsing complexity. I think
from developer's perspective it's easier to deal with a proper data structure
rather than a string. And we also have to write additional code to take care of
compatibility issues when adding fields etc. We did notice that sometimes the
launchContext become large, but need to evaluate how much load it can save if
using a file
> Continuous resource-localization for YARN containers
> ----------------------------------------------------
>
> Key: YARN-1503
> URL: https://issues.apache.org/jira/browse/YARN-1503
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Siddharth Seth
> Assignee: Jian He
> Attachments: Continuous-resource-localization.pdf
>
>
> We have a use case, where additional resources (jars, libraries etc) need to
> be made available to an already running container. Ideally, we'd like this to
> be done via YARN (instead of having potentially multiple containers per node
> download resources on their own).
> The goal is to enable NodeManagers to localize resources while container is
> running. Today, resource-localization is always the first step before
> starting a container. It will be useful if YARN can localize the resources
> continuously even while container is running.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
