[
https://issues.apache.org/jira/browse/YARN-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873523#comment-13873523
]
Hitesh Shah commented on YARN-1609:
-----------------------------------
bq. The daemon process, most likely similar in other frameworks as well,
provides critical services to tasks running on that node(for example “wireup”,
spawn user process in large numbers at once etc). In YARN, it’s hard, if not
possible, to have the those processes to be managed by YARN.
Could you add more details about what is missing/hard to do and why it cannot
be done in a performant manner in YARN today. Also, any other analysis/perf
benchmarks that you have done to make the case for the above would be useful to
look at too.
> Add Service Container type to NodeManager in YARN
> -------------------------------------------------
>
> Key: YARN-1609
> URL: https://issues.apache.org/jira/browse/YARN-1609
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager
> Affects Versions: 2.2.0
> Reporter: Wangda Tan
> Assignee: Wangda Tan
> Attachments: Add Service Container type to NodeManager in YARN-V1.pdf
>
>
> From our work to support running OpenMPI on YARN (MAPREDUCE-2911), we found
> that it’s important to have framework specific daemon process manage the
> tasks on each node directly. The daemon process, most likely similar in other
> frameworks as well, provides critical services to tasks running on that
> node(for example “wireup”, spawn user process in large numbers at once etc).
> In YARN, it’s hard, if not possible, to have the those processes to be
> managed by YARN.
> We propose to extend the container model on NodeManager side to support
> “Service Container” to run/manage such framework daemon/services process. We
> believe this is very useful to other application framework developers as well.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
