[ https://issues.apache.org/jira/browse/YARN-4758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bikas Saha updated YARN-4758: ----------------------------- Description: {color:red} This is already discussed on the umbrella JIRA YARN-1489. Copying some of my condensed summary from the design doc (section 3.2.10.3) of YARN-4692. {color} Even after the existing work in Workpreserving AM restart (Section 3.1.2 / YARN-1489), we still haven’t solved the problem of old running containers not knowing where the new AM starts running after the previous AM crashes. This is a specifically important problem to be solved for long running services where we’d like to avoid killing service containers when AMs failover. So far, we left this as a task for the apps, but solving it in YARN is much desirable. [(Task) This looks very much like service-registry (YARN-913), but for appcontainers to discover their own AMs. Combining this requirement (of any container being able to find their AM across failovers) with those of services (to be able to find through DNS where a service container is running - YARN-4757) will put our registry scalability needs to be much higher than that of just service endpoints. This calls for a more distributed solution for registry readers something that is discussed in the comments section of YARN-1489 and MAPREDUCE-6608. See comment https://issues.apache.org/jira/browse/YARN-1489?focusedCommentId=13862359&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13862359 was: {color:red} This is already discussed on the umbrella JIRA YARN-1489. Copying some of my condensed summary from the design doc (section 3.2.10.3) of YARN-4692. {color} Even after the existing work in Workpreserving AM restart (Section 3.1.2 / YARN-1489), we still haven’t solved the problem of old running containers not knowing where the new AM starts running after the previous AM crashes. This is a specifically important problem to be solved for long running services where we’d like to avoid killing service containers when AMs failover. So far, we left this as a task for the apps, but solving it in YARN is much desirable. [(Task) This looks very much like service-registry (YARN-913), but for appcontainers to discover their own AMs. Combining this requirement (of any container being able to find their AM across failovers) with those of services (to be able to find through DNS where a service container is running - YARN-4757) will put our registry scalability needs to be much higher than that of just service endpoints. This calls for a more distributed solution for registry readers something that is discussed in the comments section of YARN-1489 and MAPREDUCE-6608. > Enable discovery of AMs by containers > ------------------------------------- > > Key: YARN-4758 > URL: https://issues.apache.org/jira/browse/YARN-4758 > Project: Hadoop YARN > Issue Type: Sub-task > Reporter: Vinod Kumar Vavilapalli > > {color:red} > This is already discussed on the umbrella JIRA YARN-1489. > Copying some of my condensed summary from the design doc (section 3.2.10.3) > of YARN-4692. > {color} > Even after the existing work in Workpreserving AM restart (Section 3.1.2 / > YARN-1489), we still haven’t solved the problem of old running containers not > knowing where the new AM starts running after the previous AM crashes. This > is a specifically important problem to be solved for long running services > where we’d like to avoid killing service containers when AMs failover. So > far, we left this as a task for the apps, but solving it in YARN is much > desirable. [(Task) This looks very much like service-registry (YARN-913), > but for appcontainers to discover their own AMs. > Combining this requirement (of any container being able to find their AM > across failovers) with those of services (to be able to find through DNS > where a service container is running - YARN-4757) will put our registry > scalability needs to be much higher than that of just service endpoints. > This calls for a more distributed solution for registry readers something > that is discussed in the comments section of YARN-1489 and MAPREDUCE-6608. > See comment > https://issues.apache.org/jira/browse/YARN-1489?focusedCommentId=13862359&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13862359 -- This message was sent by Atlassian JIRA (v6.3.4#6332)