[jira] [Updated] (MAPREDUCE-4157) ResourceManager should not kill apps that are well behaved
[ https://issues.apache.org/jira/browse/MAPREDUCE-4157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans updated MAPREDUCE-4157: --- Resolution: Fixed Fix Version/s: 2.2.0-alpha 3.0.0 Status: Resolved (was: Patch Available) Thanks Jason, +1 for the change, I put this into trunk and branch-2. > ResourceManager should not kill apps that are well behaved > -- > > Key: MAPREDUCE-4157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4157 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 2.0.0-alpha >Reporter: Jason Lowe >Assignee: Jason Lowe > Fix For: 3.0.0, 2.2.0-alpha > > Attachments: MAPREDUCE-4157.patch, MAPREDUCE-4157.patch, > MAPREDUCE-4157.patch > > > Currently when the ApplicationMaster unregisters with the ResourceManager, > the RM kills (via the NMs) all the active containers for an application. > This introduces a race where the AM may be trying to clean up and may not > finish before it is killed. The RM should give the AM a chance to exit > cleanly on its own rather than always race with a pending kill on shutdown. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4157) ResourceManager should not kill apps that are well behaved
[ https://issues.apache.org/jira/browse/MAPREDUCE-4157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated MAPREDUCE-4157: -- Status: Patch Available (was: Open) > ResourceManager should not kill apps that are well behaved > -- > > Key: MAPREDUCE-4157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4157 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 2.0.0-alpha >Reporter: Jason Lowe >Assignee: Jason Lowe > Attachments: MAPREDUCE-4157.patch, MAPREDUCE-4157.patch, > MAPREDUCE-4157.patch > > > Currently when the ApplicationMaster unregisters with the ResourceManager, > the RM kills (via the NMs) all the active containers for an application. > This introduces a race where the AM may be trying to clean up and may not > finish before it is killed. The RM should give the AM a chance to exit > cleanly on its own rather than always race with a pending kill on shutdown. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4157) ResourceManager should not kill apps that are well behaved
[ https://issues.apache.org/jira/browse/MAPREDUCE-4157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated MAPREDUCE-4157: -- Attachment: MAPREDUCE-4157.patch Upmerged patch. For unmanaged AMs there is no container to wait to complete, so it skips the new FINISHING state and goes straight from RUNNING to FINISHED. > ResourceManager should not kill apps that are well behaved > -- > > Key: MAPREDUCE-4157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4157 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 2.0.0-alpha >Reporter: Jason Lowe >Assignee: Jason Lowe > Attachments: MAPREDUCE-4157.patch, MAPREDUCE-4157.patch, > MAPREDUCE-4157.patch > > > Currently when the ApplicationMaster unregisters with the ResourceManager, > the RM kills (via the NMs) all the active containers for an application. > This introduces a race where the AM may be trying to clean up and may not > finish before it is killed. The RM should give the AM a chance to exit > cleanly on its own rather than always race with a pending kill on shutdown. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4157) ResourceManager should not kill apps that are well behaved
[ https://issues.apache.org/jira/browse/MAPREDUCE-4157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated MAPREDUCE-4157: -- Target Version/s: 2.2.0-alpha (was: 2.0.0-alpha) Status: Open (was: Patch Available) Will do. Looks like the merge will be more interesting than usual due to the addition of unmanaged AMs from MAPREDUCE-4427. > ResourceManager should not kill apps that are well behaved > -- > > Key: MAPREDUCE-4157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4157 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 2.0.0-alpha >Reporter: Jason Lowe >Assignee: Jason Lowe > Attachments: MAPREDUCE-4157.patch, MAPREDUCE-4157.patch > > > Currently when the ApplicationMaster unregisters with the ResourceManager, > the RM kills (via the NMs) all the active containers for an application. > This introduces a race where the AM may be trying to clean up and may not > finish before it is killed. The RM should give the AM a chance to exit > cleanly on its own rather than always race with a pending kill on shutdown. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4157) ResourceManager should not kill apps that are well behaved
[ https://issues.apache.org/jira/browse/MAPREDUCE-4157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated MAPREDUCE-4157: -- Attachment: MAPREDUCE-4157.patch Updated patch so it applies cleanly to trunk. > ResourceManager should not kill apps that are well behaved > -- > > Key: MAPREDUCE-4157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4157 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 2.0.0-alpha >Reporter: Jason Lowe >Assignee: Jason Lowe > Attachments: MAPREDUCE-4157.patch, MAPREDUCE-4157.patch > > > Currently when the ApplicationMaster unregisters with the ResourceManager, > the RM kills (via the NMs) all the active containers for an application. > This introduces a race where the AM may be trying to clean up and may not > finish before it is killed. The RM should give the AM a chance to exit > cleanly on its own rather than always race with a pending kill on shutdown. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4157) ResourceManager should not kill apps that are well behaved
[ https://issues.apache.org/jira/browse/MAPREDUCE-4157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated MAPREDUCE-4157: -- Target Version/s: 2.0.0 Status: Patch Available (was: Open) > ResourceManager should not kill apps that are well behaved > -- > > Key: MAPREDUCE-4157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4157 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 2.0.0 >Reporter: Jason Lowe >Assignee: Jason Lowe > Attachments: MAPREDUCE-4157.patch > > > Currently when the ApplicationMaster unregisters with the ResourceManager, > the RM kills (via the NMs) all the active containers for an application. > This introduces a race where the AM may be trying to clean up and may not > finish before it is killed. The RM should give the AM a chance to exit > cleanly on its own rather than always race with a pending kill on shutdown. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4157) ResourceManager should not kill apps that are well behaved
[ https://issues.apache.org/jira/browse/MAPREDUCE-4157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated MAPREDUCE-4157: -- Attachment: MAPREDUCE-4157.patch Patch to add a FINISHING application state as described for the initial patch in MAPREDUCE-4099. Instead of transitioning from RUNNING to FINISHED, the application transitions from RUNNING to FINISHING when it unregisters with the RM. The RM then gives it so many seconds for the AM container to exit on its own. If it fails to do so, it is killed. In either case, the app moves from the FINISHING state to the FINISHED state. The timeout for the FINISHING state is the same as the liveliness monitor (i.e.: yarn.am.liveness-monitor.expiry-interval-ms) but we could easily add another property if we think these need to be configured separately. > ResourceManager should not kill apps that are well behaved > -- > > Key: MAPREDUCE-4157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4157 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 2.0.0 >Reporter: Jason Lowe >Assignee: Jason Lowe > Attachments: MAPREDUCE-4157.patch > > > Currently when the ApplicationMaster unregisters with the ResourceManager, > the RM kills (via the NMs) all the active containers for an application. > This introduces a race where the AM may be trying to clean up and may not > finish before it is killed. The RM should give the AM a chance to exit > cleanly on its own rather than always race with a pending kill on shutdown. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4157) ResourceManager should not kill apps that are well behaved
[ https://issues.apache.org/jira/browse/MAPREDUCE-4157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated MAPREDUCE-4157: -- Affects Version/s: (was: 0.23.3) 2.0.0 > ResourceManager should not kill apps that are well behaved > -- > > Key: MAPREDUCE-4157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4157 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 2.0.0 >Reporter: Jason Lowe >Assignee: Jason Lowe > > Currently when the ApplicationMaster unregisters with the ResourceManager, > the RM kills (via the NMs) all the active containers for an application. > This introduces a race where the AM may be trying to clean up and may not > finish before it is killed. The RM should give the AM a chance to exit > cleanly on its own rather than always race with a pending kill on shutdown. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
