[jira] [Commented] (YARN-7275) NM Statestore cleanup for Container updates
[ https://issues.apache.org/jira/browse/YARN-7275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16207007#comment-16207007 ] kartheek muthyala commented on YARN-7275: - Thank you [~asuresh], for the review and the commit. > NM Statestore cleanup for Container updates > --- > > Key: YARN-7275 > URL: https://issues.apache.org/jira/browse/YARN-7275 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun Suresh >Assignee: kartheek muthyala >Priority: Blocker > Fix For: 2.9.0, 3.0.0 > > Attachments: YARN-7275.001.patch, YARN-7275.002.patch, > YARN-7275.003.patch, YARN-7275.004.patch, YARN-7275.005.patch, > YARN-7275.006.patch > > > Currently, only resource updates are recorded in the NM state store, we need > to add ExecutionType updates as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7343) Add a junit test for ContainerScheduler recovery
[ https://issues.apache.org/jira/browse/YARN-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kartheek muthyala updated YARN-7343: Target Version/s: 2.9.0, 3.0.0 Fix Version/s: (was: 3.0.0) (was: 2.9.0) > Add a junit test for ContainerScheduler recovery > > > Key: YARN-7343 > URL: https://issues.apache.org/jira/browse/YARN-7343 > Project: Hadoop YARN > Issue Type: Task >Reporter: kartheek muthyala >Assignee: kartheek muthyala >Priority: Minor > > With queuing at NM, Container recovery becomes interesting. Add a junit test > for recovering containers in different states. This should test the recovery > with the ContainerScheduler class that was introduced for enabling container > queuing on contention of resources. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7343) Add a junit test for ContainerScheduler recovery
kartheek muthyala created YARN-7343: --- Summary: Add a junit test for ContainerScheduler recovery Key: YARN-7343 URL: https://issues.apache.org/jira/browse/YARN-7343 Project: Hadoop YARN Issue Type: Task Reporter: kartheek muthyala Assignee: kartheek muthyala Priority: Minor Fix For: 2.9.0, 3.0.0 With queuing at NM, Container recovery becomes interesting. Add a junit test for recovering containers in different states. This should test the recovery with the ContainerScheduler class that was introduced for enabling container queuing on contention of resources. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7275) NM Statestore cleanup for Container updates
[ https://issues.apache.org/jira/browse/YARN-7275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kartheek muthyala updated YARN-7275: Attachment: YARN-7275.006.patch Fixing the broken checkstyle and junits in YARN-7275.005.patch > NM Statestore cleanup for Container updates > --- > > Key: YARN-7275 > URL: https://issues.apache.org/jira/browse/YARN-7275 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun Suresh >Assignee: kartheek muthyala >Priority: Blocker > Attachments: YARN-7275.001.patch, YARN-7275.002.patch, > YARN-7275.003.patch, YARN-7275.004.patch, YARN-7275.005.patch, > YARN-7275.006.patch > > > Currently, only resource updates are recorded in the NM state store, we need > to add ExecutionType updates as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7275) NM Statestore cleanup for Container updates
[ https://issues.apache.org/jira/browse/YARN-7275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kartheek muthyala updated YARN-7275: Attachment: YARN-7275.005.patch [~asuresh], Can you please review the following updated patch for YARN-7275. It contains the following changes: 1. Introduced RECOVERY_COMPLETED, containerscheduler event for trying queued recovered containers. 2. Introduced RECOVER_CONTAINER_PAUSED event in ContainerEvents for moving the container that was recovered as paused from scheduled -> PAUSED after we initialize the containerimpl object for it. 3. Reused the existing event – ContainersLauncherEventType.RECOVER_CONTAINER_PAUSED for reacquiring paused container, instead of failing. + earlier patch that you are aware of. The changes only take care of recovered containers that were paused. For containers which has moved to SCHEDULED, but didn’t get a chance to run to RUNNING, the state in which they will be recovered is REQUESTED, hence ultimately they will be retried from the queue of ContainerScheduler. Let me know if the changes seems okay. > NM Statestore cleanup for Container updates > --- > > Key: YARN-7275 > URL: https://issues.apache.org/jira/browse/YARN-7275 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun Suresh >Assignee: kartheek muthyala >Priority: Blocker > Attachments: YARN-7275.001.patch, YARN-7275.002.patch, > YARN-7275.003.patch, YARN-7275.004.patch, YARN-7275.005.patch > > > Currently, only resource updates are recorded in the NM state store, we need > to add ExecutionType updates as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7275) NM Statestore cleanup for Container updates
[ https://issues.apache.org/jira/browse/YARN-7275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kartheek muthyala updated YARN-7275: Attachment: YARN-7275.004.patch Fixing Jenkin issues. The unit tests in TestContainerSchedulerQueing is passing locally. And TestDistributedScheduler issue is not related to this change. > NM Statestore cleanup for Container updates > --- > > Key: YARN-7275 > URL: https://issues.apache.org/jira/browse/YARN-7275 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun Suresh >Assignee: kartheek muthyala >Priority: Blocker > Attachments: YARN-7275.001.patch, YARN-7275.002.patch, > YARN-7275.003.patch, YARN-7275.004.patch > > > Currently, only resource updates are recorded in the NM state store, we need > to add ExecutionType updates as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7275) NM Statestore cleanup for Container updates
[ https://issues.apache.org/jira/browse/YARN-7275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kartheek muthyala updated YARN-7275: Attachment: YARN-7275.003.patch Sorry [~asuresh], couldn't reply to you earlier on this. Thank you for the quick review. I have modified the patch with the given comments. Let me know if this version is okay. > NM Statestore cleanup for Container updates > --- > > Key: YARN-7275 > URL: https://issues.apache.org/jira/browse/YARN-7275 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun Suresh >Assignee: kartheek muthyala >Priority: Blocker > Attachments: YARN-7275.001.patch, YARN-7275.002.patch, > YARN-7275.003.patch > > > Currently, only resource updates are recorded in the NM state store, we need > to add ExecutionType updates as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7275) NM Statestore cleanup for Container updates
[ https://issues.apache.org/jira/browse/YARN-7275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kartheek muthyala updated YARN-7275: Attachment: YARN-7275.002.patch Updating the patch, as it has missed commit. > NM Statestore cleanup for Container updates > --- > > Key: YARN-7275 > URL: https://issues.apache.org/jira/browse/YARN-7275 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun Suresh >Assignee: kartheek muthyala >Priority: Blocker > Attachments: YARN-7275.001.patch, YARN-7275.002.patch > > > Currently, only resource updates are recorded in the NM state store, we need > to add ExecutionType updates as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-7275) NM Statestore cleanup for Container updates
[ https://issues.apache.org/jira/browse/YARN-7275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16194198#comment-16194198 ] kartheek muthyala edited comment on YARN-7275 at 10/6/17 6:15 AM: -- Submitting the initial version of the patch that has the following changes 1. Removing from statestore that the container was queued if it is going into running state. 2. Adding container update token to statestore whenever resource is updated or exectype is updated. [~asuresh], can you please review this change. was (Author: kartheek): Submitting the initial version of the patch that has the following changes 1. Removing from statestore that the container was queued if it is going into running state. 2. Adding container update token to statestore whenever resource is updated or exectype is updated. > NM Statestore cleanup for Container updates > --- > > Key: YARN-7275 > URL: https://issues.apache.org/jira/browse/YARN-7275 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun Suresh >Assignee: kartheek muthyala >Priority: Blocker > Attachments: YARN-7275.001.patch > > > Currently, only resource updates are recorded in the NM state store, we need > to add ExecutionType updates as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7275) NM Statestore cleanup for Container updates
[ https://issues.apache.org/jira/browse/YARN-7275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kartheek muthyala updated YARN-7275: Attachment: YARN-7275.001.patch Submitting the initial version of the patch that has the following changes 1. Removing from statestore that the container was queued if it is going into running state. 2. Adding container update token to statestore whenever resource is updated or exectype is updated. > NM Statestore cleanup for Container updates > --- > > Key: YARN-7275 > URL: https://issues.apache.org/jira/browse/YARN-7275 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun Suresh >Assignee: kartheek muthyala >Priority: Blocker > Attachments: YARN-7275.001.patch > > > Currently, only resource updates are recorded in the NM state store, we need > to add ExecutionType updates as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7258) Add Node and Rack Hints to Opportunistic Scheduler
[ https://issues.apache.org/jira/browse/YARN-7258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kartheek muthyala updated YARN-7258: Attachment: YARN-7258.004.patch Updating the patch with the following change: - TestDIstributedScheduling is being removed because all the tests are already covered by TestOpportunisticContainerAllocatorAMService [~asuresh], can you please have a look at this change as well. > Add Node and Rack Hints to Opportunistic Scheduler > -- > > Key: YARN-7258 > URL: https://issues.apache.org/jira/browse/YARN-7258 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Arun Suresh >Assignee: kartheek muthyala > Attachments: YARN-7258.001.patch, YARN-7258.002.patch, > YARN-7258.003.patch, YARN-7258.004.patch > > > Currently, the Opportunistic Scheduler ignores the node and rack information > and allocates strictly on the least loaded node (based on queue length) at > the time it received the request. This JIRA is to track changes needed to > allow the OpportunisticContainerAllocator to take the node/rack name as hints. > The flow would be: > # If requested node found in the top K leastLoaded nodes, allocate on that > node > # Else, allocate on least loaded node on the same rack from the top K least > Loaded nodes. > # Else, allocate on least loaded node. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7258) Add Node and Rack Hints to Opportunistic Scheduler
[ https://issues.apache.org/jira/browse/YARN-7258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kartheek muthyala updated YARN-7258: Attachment: YARN-7258.003.patch [~asuresh], reposting the patch as I have missed some commits in the earlier one which covers some more test cases. > Add Node and Rack Hints to Opportunistic Scheduler > -- > > Key: YARN-7258 > URL: https://issues.apache.org/jira/browse/YARN-7258 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Arun Suresh >Assignee: kartheek muthyala > Attachments: YARN-7258.001.patch, YARN-7258.002.patch, > YARN-7258.003.patch > > > Currently, the Opportunistic Scheduler ignores the node and rack information > and allocates strictly on the least loaded node (based on queue length) at > the time it received the request. This JIRA is to track changes needed to > allow the OpportunisticContainerAllocator to take the node/rack name as hints. > The flow would be: > # If requested node found in the top K leastLoaded nodes, allocate on that > node > # Else, allocate on least loaded node on the same rack from the top K least > Loaded nodes. > # Else, allocate on least loaded node. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7258) Add Node and Rack Hints to Opportunistic Scheduler
[ https://issues.apache.org/jira/browse/YARN-7258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kartheek muthyala updated YARN-7258: Attachment: YARN-7258.002.patch Thank you [~asuresh] for the comment. I am uploading the next version of the patch with the changes suggested. > Add Node and Rack Hints to Opportunistic Scheduler > -- > > Key: YARN-7258 > URL: https://issues.apache.org/jira/browse/YARN-7258 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Arun Suresh >Assignee: kartheek muthyala > Attachments: YARN-7258.001.patch, YARN-7258.002.patch > > > Currently, the Opportunistic Scheduler ignores the node and rack information > and allocates strictly on the least loaded node (based on queue length) at > the time it received the request. This JIRA is to track changes needed to > allow the OpportunisticContainerAllocator to take the node/rack name as hints. > The flow would be: > # If requested node found in the top K leastLoaded nodes, allocate on that > node > # Else, allocate on least loaded node on the same rack from the top K least > Loaded nodes. > # Else, allocate on least loaded node. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7258) Add Node and Rack Hints to Opportunistic Scheduler
[ https://issues.apache.org/jira/browse/YARN-7258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kartheek muthyala updated YARN-7258: Attachment: YARN-7258.001.patch > Add Node and Rack Hints to Opportunistic Scheduler > -- > > Key: YARN-7258 > URL: https://issues.apache.org/jira/browse/YARN-7258 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Arun Suresh >Assignee: kartheek muthyala > Attachments: YARN-7258.001.patch > > > Currently, the Opportunistic Scheduler ignores the node and rack information > and allocates strictly on the least loaded node (based on queue length) at > the time it received the request. This JIRA is to track changes needed to > allow the OpportunisticContainerAllocator to take the node/rack name as hints. > The flow would be: > # If requested node found in the top K leastLoaded nodes, allocate on that > node > # Else, allocate on least loaded node on the same rack from the top K least > Loaded nodes. > # Else, allocate on least loaded node. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7240) Add more states and transitions to stabilize the NM Container state machine
[ https://issues.apache.org/jira/browse/YARN-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16180198#comment-16180198 ] kartheek muthyala commented on YARN-7240: - Thank you for [~asuresh], for rebasing and the checkstyle fixes in the patch. > Add more states and transitions to stabilize the NM Container state machine > --- > > Key: YARN-7240 > URL: https://issues.apache.org/jira/browse/YARN-7240 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Arun Suresh >Assignee: kartheek muthyala > Fix For: 2.9.0, 3.1.0 > > Attachments: YARN-7240.001.patch, YARN-7240.002.patch, > YARN-7240.002.patch > > > There seem to be a few intermediate states that can be added to improve the > stability of the NM container state machine. > For. eg: > * The REINITIALIZING should probably be split into REINITIALIZING and > REINITIALIZING_AWAITING_KILL. > * Container updates are currently handled in the ContainerScheduler, but it > would probably be better to have it plumbed through the container state > machine as a new state, say UPDATING and a new container event. > The plan is to add some extra tests too to try and test every transition. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7240) Add more states and transitions to stabilize the NM Container state machine
[ https://issues.apache.org/jira/browse/YARN-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kartheek muthyala updated YARN-7240: Attachment: YARN-7240.002.patch [~asuresh], Thank you for the quick feedback. The updated patch has the following changes 1. New testcases for TestContainerSchdedulerQueuing for the paused and promoted containers which checks the container states and their events flow through listener class. 2. Requeing the containers to the front of the queuedGuaranteed or queuedOpprotunistic queues would require to change the datastructures or we have to put use some temporary auxiliary queues to move this element. I am not sure if it can be optimal. Let me know what you think. 3. Removed the TODO comment. > Add more states and transitions to stabilize the NM Container state machine > --- > > Key: YARN-7240 > URL: https://issues.apache.org/jira/browse/YARN-7240 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Arun Suresh >Assignee: kartheek muthyala > Attachments: YARN-7240.001.patch, YARN-7240.002.patch > > > There seem to be a few intermediate states that can be added to improve the > stability of the NM container state machine. > For. eg: > * The REINITIALIZING should probably be split into REINITIALIZING and > REINITIALIZING_AWAITING_KILL. > * Container updates are currently handled in the ContainerScheduler, but it > would probably be better to have it plumbed through the container state > machine as a new state, say UPDATING and a new container event. > The plan is to add some extra tests too to try and test every transition. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-7240) Add more states and transitions to stabilize the NM Container state machine
[ https://issues.apache.org/jira/browse/YARN-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16178270#comment-16178270 ] kartheek muthyala edited comment on YARN-7240 at 9/24/17 4:55 PM: -- Attaching the initial version of the patch with the following changes: 1. Adding a new UPDATE_CONTAINER_TOKEN state to the ContainerImpl class. 2. Splitting REINITALIZING to REINITIALIZING and REINITIALIZING_AWAITING_KILL [~asuresh], please review this initial patch and let me know incase if there are any further changes. was (Author: kartheek): Attaching the initial version of the patch with the following changes: 1. Adding a new UPDATE_CONTAINER_TOKEN state to the ContainerImpl class. 2. Renaming UpdateContainerSchedulerEvent to UpdateContainerTokenEvent 3. Splitting REINITALIZING to REINITIALIZING and REINITIALIZING_AWAITING_KILL [~asuresh], please review this initial patch and let me know incase if there are any further changes. > Add more states and transitions to stabilize the NM Container state machine > --- > > Key: YARN-7240 > URL: https://issues.apache.org/jira/browse/YARN-7240 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Arun Suresh >Assignee: kartheek muthyala > Attachments: YARN-7240.001.patch > > > There seem to be a few intermediate states that can be added to improve the > stability of the NM container state machine. > For. eg: > * The REINITIALIZING should probably be split into REINITIALIZING and > REINITIALIZING_AWAITING_KILL. > * Container updates are currently handled in the ContainerScheduler, but it > would probably be better to have it plumbed through the container state > machine as a new state, say UPDATING and a new container event. > The plan is to add some extra tests too to try and test every transition. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7240) Add more states and transitions to stabilize the NM Container state machine
[ https://issues.apache.org/jira/browse/YARN-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kartheek muthyala updated YARN-7240: Attachment: YARN-7240.001.patch Attaching the initial version of the patch with the following changes: 1. Adding a new UPDATE_CONTAINER_TOKEN state to the ContainerImpl class. 2. Renaming UpdateContainerSchedulerEvent to UpdateContainerTokenEvent 3. Splitting REINITALIZING to REINITIALIZING and REINITIALIZING_AWAITING_KILL [~asuresh], please review this initial patch and let me know incase if there are any further changes. > Add more states and transitions to stabilize the NM Container state machine > --- > > Key: YARN-7240 > URL: https://issues.apache.org/jira/browse/YARN-7240 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Arun Suresh >Assignee: kartheek muthyala > Attachments: YARN-7240.001.patch > > > There seem to be a few intermediate states that can be added to improve the > stability of the NM container state machine. > For. eg: > * The REINITIALIZING should probably be split into REINITIALIZING and > REINITIALIZING_AWAITING_KILL. > * Container updates are currently handled in the ContainerScheduler, but it > would probably be better to have it plumbed through the container state > machine as a new state, say UPDATING and a new container event. > The plan is to add some extra tests too to try and test every transition. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6849) NMContainerStatus should have the Container ExecutionType.
[ https://issues.apache.org/jira/browse/YARN-6849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kartheek muthyala updated YARN-6849: Attachment: YARN-6849.002.patch Thank you [~asuresh] for the feedback. Have added Guaranteed containers into the testcase, and validated that the consumption is infact only by Guaranteed but not Opportunistic. > NMContainerStatus should have the Container ExecutionType. > -- > > Key: YARN-6849 > URL: https://issues.apache.org/jira/browse/YARN-6849 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Arun Suresh >Assignee: kartheek muthyala > Attachments: YARN-6849.001.patch, YARN-6849.002.patch > > > Currently only the ContainerState is sent to the RM in the NMContainerStatus. > This lets the restarted RM know if the container is queued or not, but It > wont know the ExecutionType. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6978) Add updateContainer API to NMClient.
[
https://issues.apache.org/jira/browse/YARN-6978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kartheek muthyala updated YARN-6978:
Attachment: YARN-6978.003.patch
Fixing the checkstyle and javac issues from the previous Jenkins report
> Add updateContainer API to NMClient.
>
>
> Key: YARN-6978
> URL: https://issues.apache.org/jira/browse/YARN-6978
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: kartheek muthyala
> Attachments: YARN-6978.001.patch, YARN-6978.002.patch,
> YARN-6978.003.patch
>
>
> This is to track the addition of updateContainer API to the {{NMClient}} and
> {{NMClientAsync}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-6978) Add updateContainer API to NMClient.
[
https://issues.apache.org/jira/browse/YARN-6978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156975#comment-16156975
]
kartheek muthyala edited comment on YARN-6978 at 9/7/17 2:01 PM:
-
[~asuresh], thank you for the quick review. Yup, that makes sense. I have
updated the patch with the suggested changes. Let me know how it looks now.
was (Author: kartheek):
[~asuresh], thanks you for the quick review. Yup, that makes sense. I have
updated the patch with the suggested changes. Let me know how it looks now.
> Add updateContainer API to NMClient.
>
>
> Key: YARN-6978
> URL: https://issues.apache.org/jira/browse/YARN-6978
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: kartheek muthyala
> Attachments: YARN-6978.001.patch, YARN-6978.002.patch
>
>
> This is to track the addition of updateContainer API to the {{NMClient}} and
> {{NMClientAsync}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6978) Add updateContainer API to NMClient.
[
https://issues.apache.org/jira/browse/YARN-6978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156975#comment-16156975
]
kartheek muthyala commented on YARN-6978:
-
[~asuresh], thanks you for the quick review. Yup, that makes sense. I have
updated the patch with the suggested changes. Let me know how it looks now.
> Add updateContainer API to NMClient.
>
>
> Key: YARN-6978
> URL: https://issues.apache.org/jira/browse/YARN-6978
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: kartheek muthyala
> Attachments: YARN-6978.001.patch, YARN-6978.002.patch
>
>
> This is to track the addition of updateContainer API to the {{NMClient}} and
> {{NMClientAsync}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6978) Add updateContainer API to NMClient.
[
https://issues.apache.org/jira/browse/YARN-6978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kartheek muthyala updated YARN-6978:
Attachment: YARN-6978.002.patch
> Add updateContainer API to NMClient.
>
>
> Key: YARN-6978
> URL: https://issues.apache.org/jira/browse/YARN-6978
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: kartheek muthyala
> Attachments: YARN-6978.001.patch, YARN-6978.002.patch
>
>
> This is to track the addition of updateContainer API to the {{NMClient}} and
> {{NMClientAsync}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6849) NMContainerStatus should have the Container ExecutionType.
[ https://issues.apache.org/jira/browse/YARN-6849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kartheek muthyala updated YARN-6849: Attachment: YARN-6849.001.patch Submitting the first version of the patch > NMContainerStatus should have the Container ExecutionType. > -- > > Key: YARN-6849 > URL: https://issues.apache.org/jira/browse/YARN-6849 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Arun Suresh >Assignee: kartheek muthyala > Attachments: YARN-6849.001.patch > > > Currently only the ContainerState is sent to the RM in the NMContainerStatus. > This lets the restarted RM know if the container is queued or not, but It > wont know the ExecutionType. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6978) Add updateContainer API to NMClient.
[
https://issues.apache.org/jira/browse/YARN-6978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kartheek muthyala updated YARN-6978:
Attachment: YARN-6978.001.patch
Attaching the first version of the patch, that introduces
updateContainerResource API on NMClient and NMClientAsync.
> Add updateContainer API to NMClient.
>
>
> Key: YARN-6978
> URL: https://issues.apache.org/jira/browse/YARN-6978
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: kartheek muthyala
> Attachments: YARN-6978.001.patch
>
>
> This is to track the addition of updateContainer API to the {{NMClient}} and
> {{NMClientAsync}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-6979) Add flag to notify all types of container updates to NM via NodeHeartbeatResponse
[
https://issues.apache.org/jira/browse/YARN-6979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16134685#comment-16134685
]
kartheek muthyala edited comment on YARN-6979 at 8/21/17 4:08 AM:
--
Thank you [~asuresh] for the review and committing this to trunk.
was (Author: kartheek):
Thank you Arun for the review and committing this to trunk.
> Add flag to notify all types of container updates to NM via
> NodeHeartbeatResponse
> -
>
> Key: YARN-6979
> URL: https://issues.apache.org/jira/browse/YARN-6979
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: kartheek muthyala
> Fix For: 2.9.0, 3.0.0-beta1
>
> Attachments: YARN-6979.001.patch, YARN-6979.002.patch,
> YARN-6979.003.patch, YARN-6979.addendum-001.patch
>
>
> Currently, only the Container Resource decrease command is sent to the NM via
> NodeHeartbeat response. This JIRA proposes to add a flag in the RM to allow
> ALL container updates (increase, decrease, promote and demote) to be
> initiated via node HB.
> The AM is still free to use the ContainerManagementPrototol's
> {{updateContainer}} API in cases where for instance, the Node HB frequency is
> very low and the AM needs to update the container as soon as possible. In
> these situations, if the Node HB arrives before the updateContainer API call,
> the call would error out, due to a version mismatch and the AM is required to
> handle it.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6979) Add flag to notify all types of container updates to NM via NodeHeartbeatResponse
[
https://issues.apache.org/jira/browse/YARN-6979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16134685#comment-16134685
]
kartheek muthyala commented on YARN-6979:
-
Thank you Arun for the review and committing this to trunk.
> Add flag to notify all types of container updates to NM via
> NodeHeartbeatResponse
> -
>
> Key: YARN-6979
> URL: https://issues.apache.org/jira/browse/YARN-6979
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: kartheek muthyala
> Fix For: 2.9.0, 3.0.0-beta1
>
> Attachments: YARN-6979.001.patch, YARN-6979.002.patch,
> YARN-6979.003.patch, YARN-6979.addendum-001.patch
>
>
> Currently, only the Container Resource decrease command is sent to the NM via
> NodeHeartbeat response. This JIRA proposes to add a flag in the RM to allow
> ALL container updates (increase, decrease, promote and demote) to be
> initiated via node HB.
> The AM is still free to use the ContainerManagementPrototol's
> {{updateContainer}} API in cases where for instance, the Node HB frequency is
> very low and the AM needs to update the container as soon as possible. In
> these situations, if the Node HB arrives before the updateContainer API call,
> the call would error out, due to a version mismatch and the AM is required to
> handle it.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6979) Add flag to notify all types of container updates to NM via NodeHeartbeatResponse
[
https://issues.apache.org/jira/browse/YARN-6979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kartheek muthyala updated YARN-6979:
Attachment: YARN-6979.addendum-001.patch
Attaching the addendum patch for renaming CMgrDecreaseContainersResourceEvent
to CMgrUpdateContainersEvent and changing DECREASE_CONTAINERS_RESOURCE to
UPDATE_RESOURCE
> Add flag to notify all types of container updates to NM via
> NodeHeartbeatResponse
> -
>
> Key: YARN-6979
> URL: https://issues.apache.org/jira/browse/YARN-6979
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: kartheek muthyala
> Fix For: 2.9.0, 3.0.0-beta1
>
> Attachments: YARN-6979.001.patch, YARN-6979.002.patch,
> YARN-6979.003.patch, YARN-6979.addendum-001.patch
>
>
> Currently, only the Container Resource decrease command is sent to the NM via
> NodeHeartbeat response. This JIRA proposes to add a flag in the RM to allow
> ALL container updates (increase, decrease, promote and demote) to be
> initiated via node HB.
> The AM is still free to use the ContainerManagementPrototol's
> {{updateContainer}} API in cases where for instance, the Node HB frequency is
> very low and the AM needs to update the container as soon as possible. In
> these situations, if the Node HB arrives before the updateContainer API call,
> the call would error out, due to a version mismatch and the AM is required to
> handle it.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6979) Add flag to allow all container updates to be initiated via NodeHeartbeatResponse
[
https://issues.apache.org/jira/browse/YARN-6979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16134410#comment-16134410
]
kartheek muthyala commented on YARN-6979:
-
[~asuresh], I have fixed the unit test according to your suggestion. The
failing unit tests in the Hadoop report seems to be unrelated to this patch and
are failing in trunk too. Taken care of the checkstyle and whitespace issues in
the last patch too.
> Add flag to allow all container updates to be initiated via
> NodeHeartbeatResponse
> -
>
> Key: YARN-6979
> URL: https://issues.apache.org/jira/browse/YARN-6979
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: kartheek muthyala
> Attachments: YARN-6979.001.patch, YARN-6979.002.patch,
> YARN-6979.003.patch
>
>
> Currently, only the Container Resource decrease command is sent to the NM via
> NodeHeartbeat response. This JIRA proposes to add a flag in the RM to allow
> ALL container updates (increase, decrease, promote and demote) to be
> initiated via node HB.
> The AM is still free to use the ContainerManagementPrototol's
> {{updateContainer}} API in cases where for instance, the Node HB frequency is
> very low and the AM needs to update the container as soon as possible. In
> these situations, if the Node HB arrives before the updateContainer API call,
> the call would error out, due to a version mismatch and the AM is required to
> handle it.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6979) Add flag to allow all container updates to be initiated via NodeHeartbeatResponse
[
https://issues.apache.org/jira/browse/YARN-6979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kartheek muthyala updated YARN-6979:
Attachment: YARN-6979.003.patch
On top of the last version of the patch, this patch enhances
testContainerAutoUpdateContainer with promotion, demotion, increase and
decrease logic
> Add flag to allow all container updates to be initiated via
> NodeHeartbeatResponse
> -
>
> Key: YARN-6979
> URL: https://issues.apache.org/jira/browse/YARN-6979
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: kartheek muthyala
> Attachments: YARN-6979.001.patch, YARN-6979.002.patch,
> YARN-6979.003.patch
>
>
> Currently, only the Container Resource decrease command is sent to the NM via
> NodeHeartbeat response. This JIRA proposes to add a flag in the RM to allow
> ALL container updates (increase, decrease, promote and demote) to be
> initiated via node HB.
> The AM is still free to use the ContainerManagementPrototol's
> {{updateContainer}} API in cases where for instance, the Node HB frequency is
> very low and the AM needs to update the container as soon as possible. In
> these situations, if the Node HB arrives before the updateContainer API call,
> the call would error out, due to a version mismatch and the AM is required to
> handle it.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6979) Add flag to allow all container updates to be initiated via NodeHeartbeatResponse
[
https://issues.apache.org/jira/browse/YARN-6979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kartheek muthyala updated YARN-6979:
Attachment: YARN-6979.002.patch
> Add flag to allow all container updates to be initiated via
> NodeHeartbeatResponse
> -
>
> Key: YARN-6979
> URL: https://issues.apache.org/jira/browse/YARN-6979
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: kartheek muthyala
> Attachments: YARN-6979.001.patch, YARN-6979.002.patch
>
>
> Currently, only the Container Resource decrease command is sent to the NM via
> NodeHeartbeat response. This JIRA proposes to add a flag in the RM to allow
> ALL container updates (increase, decrease, promote and demote) to be
> initiated via node HB.
> The AM is still free to use the ContainerManagementPrototol's
> {{updateContainer}} API in cases where for instance, the Node HB frequency is
> very low and the AM needs to update the container as soon as possible. In
> these situations, if the Node HB arrives before the updateContainer API call,
> the call would error out, due to a version mismatch and the AM is required to
> handle it.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6979) Add flag to allow all container updates to be initiated via NodeHeartbeatResponse
[
https://issues.apache.org/jira/browse/YARN-6979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kartheek muthyala updated YARN-6979:
Attachment: YARN-6979.001.patch
Attaching the initial version of the patch to get the Hadoop Bug report
> Add flag to allow all container updates to be initiated via
> NodeHeartbeatResponse
> -
>
> Key: YARN-6979
> URL: https://issues.apache.org/jira/browse/YARN-6979
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: kartheek muthyala
> Attachments: YARN-6979.001.patch
>
>
> Currently, only the Container Resource increase command is sent to the NM via
> NodeHeartbeat response. This JIRA proposes to add a flag in the RM to allow
> ALL container updates (increase, decrease, promote and demote) to initiated
> via node HB.
> The AM is still free to use the ContainerManagementPrototol's
> {{updateContainer}} API in cases where for instance, the Node HB frequency is
> very low and the AM needs to update the container as soon as possible. In
> these situations, if the Node HB arrives before the updateContainer API call,
> the call would error out, due to a version mismatch and the AM is required to
> handle it.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5978) ContainerScheduler and Container state machine changes to support ExecType update
[ https://issues.apache.org/jira/browse/YARN-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16125587#comment-16125587 ] kartheek muthyala commented on YARN-5978: - [~asuresh] FYI, The failed unit tests are not related to this change. > ContainerScheduler and Container state machine changes to support ExecType > update > - > > Key: YARN-5978 > URL: https://issues.apache.org/jira/browse/YARN-5978 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun Suresh >Assignee: kartheek muthyala > Attachments: YARN-5978.001.patch, YARN-5978.002.patch, > YARN-5978.003.patch > > > ContainerScheduler should support updateContainer API for > - Container Resource update > - ExecType update that can change an opportunistic to guaranteed and > vice-versa > Adding a new ContainerState event, UpdateContainerStateEvent to support > UPDATE_CONTAINER call from RM. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5978) ContainerScheduler and Container state machine changes to support ExecType update
[ https://issues.apache.org/jira/browse/YARN-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16125437#comment-16125437 ] kartheek muthyala commented on YARN-5978: - Hi [~asuresh], Thanks for the comments. I have taken care of your concerns. Please have a look at the latest patch. > ContainerScheduler and Container state machine changes to support ExecType > update > - > > Key: YARN-5978 > URL: https://issues.apache.org/jira/browse/YARN-5978 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun Suresh >Assignee: kartheek muthyala > Attachments: YARN-5978.001.patch, YARN-5978.002.patch, > YARN-5978.003.patch > > > ContainerScheduler should support updateContainer API for > - Container Resource update > - ExecType update that can change an opportunistic to guaranteed and > vice-versa > Adding a new ContainerState event, UpdateContainerStateEvent to support > UPDATE_CONTAINER call from RM. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5978) ContainerScheduler and Container state machine changes to support ExecType update
[ https://issues.apache.org/jira/browse/YARN-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kartheek muthyala updated YARN-5978: Attachment: YARN-5978.003.patch > ContainerScheduler and Container state machine changes to support ExecType > update > - > > Key: YARN-5978 > URL: https://issues.apache.org/jira/browse/YARN-5978 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun Suresh >Assignee: kartheek muthyala > Attachments: YARN-5978.001.patch, YARN-5978.002.patch, > YARN-5978.003.patch > > > ContainerScheduler should support updateContainer API for > - Container Resource update > - ExecType update that can change an opportunistic to guaranteed and > vice-versa > Adding a new ContainerState event, UpdateContainerStateEvent to support > UPDATE_CONTAINER call from RM. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5978) ContainerScheduler and Container state machine changes to support ExecType update
[ https://issues.apache.org/jira/browse/YARN-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kartheek muthyala updated YARN-5978: Attachment: YARN-5978.002.patch Fixing checkstyle, Javadoc and broken junit tests. > ContainerScheduler and Container state machine changes to support ExecType > update > - > > Key: YARN-5978 > URL: https://issues.apache.org/jira/browse/YARN-5978 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun Suresh >Assignee: kartheek muthyala > Attachments: YARN-5978.001.patch, YARN-5978.002.patch > > > ContainerScheduler should support updateContainer API for > - Container Resource update > - ExecType update that can change an opportunistic to guaranteed and > vice-versa > Adding a new ContainerState event, UpdateContainerStateEvent to support > UPDATE_CONTAINER call from RM. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5978) ContainerScheduler and Container state machine changes to support ExecType update
[ https://issues.apache.org/jira/browse/YARN-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kartheek muthyala updated YARN-5978: Attachment: YARN-5978.001.patch Submitting the initial version of the patch to figure out javac, findbugs, Javadoc and other issues > ContainerScheduler and Container state machine changes to support ExecType > update > - > > Key: YARN-5978 > URL: https://issues.apache.org/jira/browse/YARN-5978 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun Suresh >Assignee: kartheek muthyala > Attachments: YARN-5978.001.patch > > > ContainerScheduler should support updateContainer API for > - Container Resource update > - ExecType update that can change an opportunistic to guaranteed and > vice-versa > Adding a new ContainerState event, UpdateContainerStateEvent to support > UPDATE_CONTAINER call from RM. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5978) ContainerScheduler and Container state machine changes to support ExecType update
[ https://issues.apache.org/jira/browse/YARN-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kartheek muthyala updated YARN-5978: Description: ContainerScheduler should support updateContainer API for - Container Resource update - ExecType update that can change an opportunistic to guaranteed and vice-versa Adding a new ContainerState event, UpdateContainerStateEvent to support UPDATE_CONTAINER call from RM. > ContainerScheduler and Container state machine changes to support ExecType > update > - > > Key: YARN-5978 > URL: https://issues.apache.org/jira/browse/YARN-5978 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun Suresh >Assignee: kartheek muthyala > > ContainerScheduler should support updateContainer API for > - Container Resource update > - ExecType update that can change an opportunistic to guaranteed and > vice-versa > Adding a new ContainerState event, UpdateContainerStateEvent to support > UPDATE_CONTAINER call from RM. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5977) ContainerManagementProtocol changes to support change of container ExecutionType
[
https://issues.apache.org/jira/browse/YARN-5977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kartheek muthyala updated YARN-5977:
Attachment: YARN-5977.006.patch
> ContainerManagementProtocol changes to support change of container
> ExecutionType
>
>
> Key: YARN-5977
> URL: https://issues.apache.org/jira/browse/YARN-5977
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: kartheek muthyala
> Attachments: YARN-5977.001.patch, YARN-5977.002.patch,
> YARN-5977.003.patch, YARN-5977.004.patch, YARN-5977.005.patch,
> YARN-5977.006.patch
>
>
> JIRA to track the following changes:
> * Changes in the ContainerManagementProtocol - add an {{updateContainer()}}
> method.
> * Add the new Request and Response Objects and their corresponding PBImpl
> classes.
> * Add deprecate attribute to {{increaseContainersResouce()}} method - since
> this functionality will be subsumed by {{updateContainer()}}
> * Changes in NMClient to deprecate increaseContainer methods and route all
> calls through the new updateContainer API
> * On the NM side, route increaseContainer calls to the new updateContainer
> method.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5977) ContainerManagementProtocol changes to support change of container ExecutionType
[
https://issues.apache.org/jira/browse/YARN-5977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kartheek muthyala updated YARN-5977:
Attachment: (was: YARN-5977.005.patch)
> ContainerManagementProtocol changes to support change of container
> ExecutionType
>
>
> Key: YARN-5977
> URL: https://issues.apache.org/jira/browse/YARN-5977
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: kartheek muthyala
> Attachments: YARN-5977.001.patch, YARN-5977.002.patch,
> YARN-5977.003.patch, YARN-5977.004.patch, YARN-5977.005.patch
>
>
> JIRA to track the following changes:
> * Changes in the ContainerManagementProtocol - add an {{updateContainer()}}
> method.
> * Add the new Request and Response Objects and their corresponding PBImpl
> classes.
> * Add deprecate attribute to {{increaseContainersResouce()}} method - since
> this functionality will be subsumed by {{updateContainer()}}
> * Changes in NMClient to deprecate increaseContainer methods and route all
> calls through the new updateContainer API
> * On the NM side, route increaseContainer calls to the new updateContainer
> method.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5977) ContainerManagementProtocol changes to support change of container ExecutionType
[
https://issues.apache.org/jira/browse/YARN-5977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kartheek muthyala updated YARN-5977:
Attachment: YARN-5977.005.patch
> ContainerManagementProtocol changes to support change of container
> ExecutionType
>
>
> Key: YARN-5977
> URL: https://issues.apache.org/jira/browse/YARN-5977
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: kartheek muthyala
> Attachments: YARN-5977.001.patch, YARN-5977.002.patch,
> YARN-5977.003.patch, YARN-5977.004.patch, YARN-5977.005.patch
>
>
> JIRA to track the following changes:
> * Changes in the ContainerManagementProtocol - add an {{updateContainer()}}
> method.
> * Add the new Request and Response Objects and their corresponding PBImpl
> classes.
> * Add deprecate attribute to {{increaseContainersResouce()}} method - since
> this functionality will be subsumed by {{updateContainer()}}
> * Changes in NMClient to deprecate increaseContainer methods and route all
> calls through the new updateContainer API
> * On the NM side, route increaseContainer calls to the new updateContainer
> method.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5977) ContainerManagementProtocol changes to support change of container ExecutionType
[
https://issues.apache.org/jira/browse/YARN-5977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kartheek muthyala updated YARN-5977:
Attachment: YARN-5977.005.patch
> ContainerManagementProtocol changes to support change of container
> ExecutionType
>
>
> Key: YARN-5977
> URL: https://issues.apache.org/jira/browse/YARN-5977
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: kartheek muthyala
> Attachments: YARN-5977.001.patch, YARN-5977.002.patch,
> YARN-5977.003.patch, YARN-5977.004.patch, YARN-5977.005.patch
>
>
> JIRA to track the following changes:
> * Changes in the ContainerManagementProtocol - add an {{updateContainer()}}
> method.
> * Add the new Request and Response Objects and their corresponding PBImpl
> classes.
> * Add deprecate attribute to {{increaseContainersResouce()}} method - since
> this functionality will be subsumed by {{updateContainer()}}
> * Changes in NMClient to deprecate increaseContainer methods and route all
> calls through the new updateContainer API
> * On the NM side, route increaseContainer calls to the new updateContainer
> method.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5977) ContainerManagementProtocol changes to support change of container ExecutionType
[
https://issues.apache.org/jira/browse/YARN-5977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kartheek muthyala updated YARN-5977:
Attachment: YARN-5977.004.patch
Fixing Javac warnings.
> ContainerManagementProtocol changes to support change of container
> ExecutionType
>
>
> Key: YARN-5977
> URL: https://issues.apache.org/jira/browse/YARN-5977
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: kartheek muthyala
> Attachments: YARN-5977.001.patch, YARN-5977.002.patch,
> YARN-5977.003.patch, YARN-5977.004.patch
>
>
> JIRA to track the following changes:
> * Changes in the ContainerManagementProtocol - add an {{updateContainer()}}
> method.
> * Add the new Request and Response Objects and their corresponding PBImpl
> classes.
> * Add deprecate attribute to {{increaseContainersResouce()}} method - since
> this functionality will be subsumed by {{updateContainer()}}
> * Changes in NMClient to deprecate increaseContainer methods and route all
> calls through the new updateContainer API
> * On the NM side, route increaseContainer calls to the new updateContainer
> method.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5977) ContainerManagementProtocol changes to support change of container ExecutionType
[
https://issues.apache.org/jira/browse/YARN-5977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16107421#comment-16107421
]
kartheek muthyala commented on YARN-5977:
-
Unrelated unittest failed
hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppStarvation.
Resubmitting the patch to see if it is some transient issue.
> ContainerManagementProtocol changes to support change of container
> ExecutionType
>
>
> Key: YARN-5977
> URL: https://issues.apache.org/jira/browse/YARN-5977
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: kartheek muthyala
> Attachments: YARN-5977.001.patch, YARN-5977.002.patch,
> YARN-5977.003.patch
>
>
> JIRA to track the following changes:
> * Changes in the ContainerManagementProtocol - add an {{updateContainer()}}
> method.
> * Add the new Request and Response Objects and their corresponding PBImpl
> classes.
> * Add deprecate attribute to {{increaseContainersResouce()}} method - since
> this functionality will be subsumed by {{updateContainer()}}
> * Changes in NMClient to deprecate increaseContainer methods and route all
> calls through the new updateContainer API
> * On the NM side, route increaseContainer calls to the new updateContainer
> method.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5977) ContainerManagementProtocol changes to support change of container ExecutionType
[
https://issues.apache.org/jira/browse/YARN-5977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kartheek muthyala updated YARN-5977:
Attachment: YARN-5977.003.patch
> ContainerManagementProtocol changes to support change of container
> ExecutionType
>
>
> Key: YARN-5977
> URL: https://issues.apache.org/jira/browse/YARN-5977
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: kartheek muthyala
> Attachments: YARN-5977.001.patch, YARN-5977.002.patch,
> YARN-5977.003.patch
>
>
> JIRA to track the following changes:
> * Changes in the ContainerManagementProtocol - add an {{updateContainer()}}
> method.
> * Add the new Request and Response Objects and their corresponding PBImpl
> classes.
> * Add deprecate attribute to {{increaseContainersResouce()}} method - since
> this functionality will be subsumed by {{updateContainer()}}
> * Changes in NMClient to deprecate increaseContainer methods and route all
> calls through the new updateContainer API
> * On the NM side, route increaseContainer calls to the new updateContainer
> method.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5977) ContainerManagementProtocol changes to support change of container ExecutionType
[
https://issues.apache.org/jira/browse/YARN-5977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kartheek muthyala updated YARN-5977:
Attachment: YARN-5977.002.patch
Fixed failing Junits, CheckStyle, Javadoc and findbugs issues.
> ContainerManagementProtocol changes to support change of container
> ExecutionType
>
>
> Key: YARN-5977
> URL: https://issues.apache.org/jira/browse/YARN-5977
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: kartheek muthyala
> Attachments: YARN-5977.001.patch, YARN-5977.002.patch
>
>
> JIRA to track the following changes:
> * Changes in the ContainerManagementProtocol - add an {{updateContainer()}}
> method.
> * Add the new Request and Response Objects and their corresponding PBImpl
> classes.
> * Add deprecate attribute to {{increaseContainersResouce()}} method - since
> this functionality will be subsumed by {{updateContainer()}}
> * Changes in NMClient to deprecate increaseContainer methods and route all
> calls through the new updateContainer API
> * On the NM side, route increaseContainer calls to the new updateContainer
> method.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-5977) ContainerManagementProtocol changes to support change of container ExecutionType
[ https://issues.apache.org/jira/browse/YARN-5977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16106514#comment-16106514 ] kartheek muthyala edited comment on YARN-5977 at 7/30/17 2:11 PM: -- Submitting the initial version of the patch which introduces ContainerUpdateRequestProto and ContainerUpdateResponseProto - an API that can update the resources of running containers.on the nodemanager. was (Author: kartheek): Submitting the initial version of the patch which introduces ContainerUpdateRequestProto and ContainerUpdateResponseProto. > ContainerManagementProtocol changes to support change of container > ExecutionType > > > Key: YARN-5977 > URL: https://issues.apache.org/jira/browse/YARN-5977 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun Suresh >Assignee: kartheek muthyala > Attachments: YARN-5977.001.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5977) ContainerManagementProtocol changes to support change of container ExecutionType
[ https://issues.apache.org/jira/browse/YARN-5977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kartheek muthyala updated YARN-5977: Attachment: YARN-5977.001.patch Submitting the initial version of the patch which introduces ContainerUpdateRequestProto and ContainerUpdateResponseProto. > ContainerManagementProtocol changes to support change of container > ExecutionType > > > Key: YARN-5977 > URL: https://issues.apache.org/jira/browse/YARN-5977 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun Suresh >Assignee: kartheek muthyala > Attachments: YARN-5977.001.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
