[ https://issues.apache.org/jira/browse/YARN-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13875418#comment-13875418 ]
Hitesh Shah commented on YARN-1609: ----------------------------------- [~lvm] I think I understand what the goals are though I am still unclear on the specifics as to what the real problems were when trying to launch processes directly via the NM as compared to using the service container. - Is the NM flow too slow for launching a new container or killing one? - Is it difficult to monitoring of the status of a container? - Can the NM not handle launching/managing a large no. of containers on a single machine? It would also be helpful to understand what kind of workarounds/hacks you needed to do so as to understand the underlying problems that you faced. > Add Service Container type to NodeManager in YARN > ------------------------------------------------- > > Key: YARN-1609 > URL: https://issues.apache.org/jira/browse/YARN-1609 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager > Affects Versions: 2.2.0 > Reporter: Wangda Tan > Assignee: Wangda Tan > Attachments: Add Service Container type to NodeManager in YARN-V1.pdf > > > From our work to support running OpenMPI on YARN (MAPREDUCE-2911), we found > that it’s important to have framework specific daemon process manage the > tasks on each node directly. The daemon process, most likely similar in other > frameworks as well, provides critical services to tasks running on that > node(for example “wireup”, spawn user process in large numbers at once etc). > In YARN, it’s hard, if not possible, to have the those processes to be > managed by YARN. > We propose to extend the container model on NodeManager side to support > “Service Container” to run/manage such framework daemon/services process. We > believe this is very useful to other application framework developers as well. -- This message was sent by Atlassian JIRA (v6.1.5#6160)