[jira] [Updated] (YARN-417) Add a poller that allows the AM to receive notifications when it is assigned containers
[ https://issues.apache.org/jira/browse/YARN-417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-417: Attachment: YARN-417-6.patch Updated patch makes the changes suggested in Bikas's comment, including the rebase. For the while loop, I moved things around a little in a way that seems more clear to me. > Add a poller that allows the AM to receive notifications when it is assigned > containers > --- > > Key: YARN-417 > URL: https://issues.apache.org/jira/browse/YARN-417 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, applications >Affects Versions: 2.0.3-alpha >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Attachments: AMRMClientAsync-1.java, AMRMClientAsync.java, > YARN-417-1.patch, YARN-417-2.patch, YARN-417-3.patch, YARN-417-4.patch, > YARN-417-4.patch, YARN-417-5.patch, YARN-417-6.patch, YARN-417.patch, > YarnAppMaster.java, YarnAppMasterListener.java > > > Writing AMs would be easier for some if they did not have to handle > heartbeating to the RM on their own. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-417) Add a poller that allows the AM to receive notifications when it is assigned containers
[ https://issues.apache.org/jira/browse/YARN-417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-417: Attachment: YARN-417-5.patch Updated patch * adds @Unstable and @Evolving tags to AMRMClientAsync * synchronizes on client in unregisterApplicationMaster so that no heartbeats will be sent after unregistering * moves client.stop() in stop so that the AMRMClient will not be stopped until the heartbeat thread has been joined with * fixes potential flakiness in the test * updates comments to reflect AMRMClientAsync's expected usage > Add a poller that allows the AM to receive notifications when it is assigned > containers > --- > > Key: YARN-417 > URL: https://issues.apache.org/jira/browse/YARN-417 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, applications >Affects Versions: 2.0.3-alpha >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Attachments: AMRMClientAsync-1.java, AMRMClientAsync.java, > YARN-417-1.patch, YARN-417-2.patch, YARN-417-3.patch, YARN-417-4.patch, > YARN-417-4.patch, YARN-417-5.patch, YARN-417.patch, YarnAppMaster.java, > YarnAppMasterListener.java > > > Writing AMs would be easier for some if they did not have to handle > heartbeating to the RM on their own. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-417) Add a poller that allows the AM to receive notifications when it is assigned containers
[ https://issues.apache.org/jira/browse/YARN-417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-417: Attachment: YARN-417-4.patch Updated patch uses interrupt to fix the deadlock that was causing the test to fail > Add a poller that allows the AM to receive notifications when it is assigned > containers > --- > > Key: YARN-417 > URL: https://issues.apache.org/jira/browse/YARN-417 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, applications >Affects Versions: 2.0.3-alpha >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Attachments: AMRMClientAsync-1.java, AMRMClientAsync.java, > YARN-417-1.patch, YARN-417-2.patch, YARN-417-3.patch, YARN-417-4.patch, > YARN-417-4.patch, YARN-417.patch, YarnAppMaster.java, > YarnAppMasterListener.java > > > Writing AMs would be easier for some if they did not have to handle > heartbeating to the RM on their own. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-417) Add a poller that allows the AM to receive notifications when it is assigned containers
[ https://issues.apache.org/jira/browse/YARN-417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-417: Attachment: YARN-417-4.patch > Add a poller that allows the AM to receive notifications when it is assigned > containers > --- > > Key: YARN-417 > URL: https://issues.apache.org/jira/browse/YARN-417 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, applications >Affects Versions: 2.0.3-alpha >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Attachments: AMRMClientAsync-1.java, AMRMClientAsync.java, > YARN-417-1.patch, YARN-417-2.patch, YARN-417-3.patch, YARN-417-4.patch, > YARN-417.patch, YarnAppMaster.java, YarnAppMasterListener.java > > > Writing AMs would be easier for some if they did not have to handle > heartbeating to the RM on their own. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-417) Add a poller that allows the AM to receive notifications when it is assigned containers
[ https://issues.apache.org/jira/browse/YARN-417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-417: Attachment: YARN-417-3.patch Latest patch includes a simple test with a mocked AMRMClient > Add a poller that allows the AM to receive notifications when it is assigned > containers > --- > > Key: YARN-417 > URL: https://issues.apache.org/jira/browse/YARN-417 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, applications >Affects Versions: 2.0.3-alpha >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Attachments: AMRMClientAsync-1.java, AMRMClientAsync.java, > YARN-417-1.patch, YARN-417-2.patch, YARN-417-3.patch, YARN-417.patch, > YarnAppMaster.java, YarnAppMasterListener.java > > > Writing AMs would be easier for some if they did not have to handle > heartbeating to the RM on their own. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-417) Add a poller that allows the AM to receive notifications when it is assigned containers
[ https://issues.apache.org/jira/browse/YARN-417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-417: Attachment: YARN-417-2.patch Updated patch includes changes mentioned in above comment > Add a poller that allows the AM to receive notifications when it is assigned > containers > --- > > Key: YARN-417 > URL: https://issues.apache.org/jira/browse/YARN-417 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, applications >Affects Versions: 2.0.3-alpha >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Attachments: AMRMClientAsync-1.java, AMRMClientAsync.java, > YARN-417-1.patch, YARN-417-2.patch, YARN-417.patch, YarnAppMaster.java, > YarnAppMasterListener.java > > > Writing AMs would be easier for some if they did not have to handle > heartbeating to the RM on their own. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-417) Add a poller that allows the AM to receive notifications when it is assigned containers
[ https://issues.apache.org/jira/browse/YARN-417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-417: Attachment: YARN-417-1.patch > Add a poller that allows the AM to receive notifications when it is assigned > containers > --- > > Key: YARN-417 > URL: https://issues.apache.org/jira/browse/YARN-417 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, applications >Affects Versions: 2.0.3-alpha >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Attachments: AMRMClientAsync-1.java, AMRMClientAsync.java, > YARN-417-1.patch, YARN-417.patch, YarnAppMaster.java, > YarnAppMasterListener.java > > > Writing AMs would be easier for some if they did not have to handle > heartbeating to the RM on their own. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-417) Add a poller that allows the AM to receive notifications when it is assigned containers
[ https://issues.apache.org/jira/browse/YARN-417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-417: Attachment: AMRMClientAsync-1.java > Add a poller that allows the AM to receive notifications when it is assigned > containers > --- > > Key: YARN-417 > URL: https://issues.apache.org/jira/browse/YARN-417 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, applications >Affects Versions: 2.0.3-alpha >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Attachments: AMRMClientAsync-1.java, AMRMClientAsync.java, > YARN-417.patch, YarnAppMaster.java, YarnAppMasterListener.java > > > Writing AMs would be easier for some if they did not have to handle > heartbeating to the RM on their own. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-417) Add a poller that allows the AM to receive notifications when it is assigned containers
[ https://issues.apache.org/jira/browse/YARN-417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated YARN-417: Attachment: AMRMClientAsync.java Attaching an interface proposal > Add a poller that allows the AM to receive notifications when it is assigned > containers > --- > > Key: YARN-417 > URL: https://issues.apache.org/jira/browse/YARN-417 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, applications >Affects Versions: 2.0.3-alpha >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Attachments: AMRMClientAsync.java, YARN-417.patch, > YarnAppMaster.java, YarnAppMasterListener.java > > > Writing AMs would be easier for some if they did not have to handle > heartbeating to the RM on their own. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-417) Add a poller that allows the AM to receive notifications when it is assigned containers
[ https://issues.apache.org/jira/browse/YARN-417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Riccomini updated YARN-417: - Attachment: YarnAppMasterListener.java YarnAppMaster.java Hey Guys, I'm attaching two files, to provide some perspective. This is how I handle dealing with the YARN AM, right now. Sandy, I think your patch is following a similar idea, but models a little bit less of the lifecycle. One open question, regarding how I've set things up, is whether it makes sense to just have onContainerComplete, as it is in the patch, or have onContainerLost/onContainerFail/onContainerSuccess. I think the latter makes thins easier, since YARN has some pre-defined return codes (e.g. -100) that are a bit confusing to deal with if you don't read the code/Javadocs. Lastly, my patch isn't using the AMRMClient patch that was recently added - it's using the raw API plus my own internal "application master helper". Should probably be converted to use the new AMRMClient. Cheers, Chris > Add a poller that allows the AM to receive notifications when it is assigned > containers > --- > > Key: YARN-417 > URL: https://issues.apache.org/jira/browse/YARN-417 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, applications >Affects Versions: 2.0.3-alpha >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Attachments: YARN-417.patch, YarnAppMaster.java, > YarnAppMasterListener.java > > > Writing AMs would be easier for some if they did not have to handle > heartbeating to the RM on their own. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-417) Add a poller that allows the AM to receive notifications when it is assigned containers
[ https://issues.apache.org/jira/browse/YARN-417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-417: Issue Type: Sub-task (was: New Feature) Parent: YARN-418 > Add a poller that allows the AM to receive notifications when it is assigned > containers > --- > > Key: YARN-417 > URL: https://issues.apache.org/jira/browse/YARN-417 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, applications >Affects Versions: 2.0.3-alpha >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Attachments: YARN-417.patch > > > Writing AMs would be easier for some if they did not have to handle > heartbeating to the RM on their own. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-417) Add a poller that allows the AM to receive notifications when it is assigned containers
[ https://issues.apache.org/jira/browse/YARN-417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-417: Attachment: YARN-417.patch > Add a poller that allows the AM to receive notifications when it is assigned > containers > --- > > Key: YARN-417 > URL: https://issues.apache.org/jira/browse/YARN-417 > Project: Hadoop YARN > Issue Type: New Feature > Components: api, applications >Affects Versions: 2.0.3-alpha >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Attachments: YARN-417.patch > > > Writing AMs would be easier for some if they did not have to handle > heartbeating to the RM on their own. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
