[jira] [Updated] (YARN-4412) Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers
[ https://issues.apache.org/jira/browse/YARN-4412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Subru Krishnan updated YARN-4412: - Fix Version/s: (was: 3.0.0) 3.0.0-alpha1 > Create ClusterMonitor to compute ordered list of preferred NMs for > OPPORTUNITIC containers > -- > > Key: YARN-4412 > URL: https://issues.apache.org/jira/browse/YARN-4412 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Arun Suresh >Assignee: Arun Suresh > Fix For: 2.9.0, 3.0.0-alpha1 > > Attachments: YARN-4412-yarn-2877.v1.patch, > YARN-4412-yarn-2877.v2.patch, YARN-4412-yarn-2877.v3.patch, > YARN-4412-yarn-2877.v4.patch, YARN-4412-yarn-2877.v5.patch, > YARN-4412-yarn-2877.v6.patch, YARN-4412.007.patch, YARN-4412.008.patch, > YARN-4412.009.patch, YARN-4412.addendum-001.patch, YARN-4412.find-bugs.patch > > > Introduce a Cluster Monitor that aggregates load information from individual > Node Managers and computes an ordered list of preferred Node managers to be > used as target Nodes for OPPORTUNISTIC container allocations. > This list can be pushed out to the Node Manager (specifically the AMRMProxy > running on the Node) via the Allocate Response. This will be used to make > local Scheduling decisions -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-4412) Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers
[ https://issues.apache.org/jira/browse/YARN-4412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-4412: -- Fix Version/s: (was: 3.0.0-alpha1) 2.9.0 > Create ClusterMonitor to compute ordered list of preferred NMs for > OPPORTUNITIC containers > -- > > Key: YARN-4412 > URL: https://issues.apache.org/jira/browse/YARN-4412 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Arun Suresh >Assignee: Arun Suresh > Fix For: 2.9.0 > > Attachments: YARN-4412-yarn-2877.v1.patch, > YARN-4412-yarn-2877.v2.patch, YARN-4412-yarn-2877.v3.patch, > YARN-4412-yarn-2877.v4.patch, YARN-4412-yarn-2877.v5.patch, > YARN-4412-yarn-2877.v6.patch, YARN-4412.007.patch, YARN-4412.008.patch, > YARN-4412.009.patch, YARN-4412.addendum-001.patch, YARN-4412.find-bugs.patch > > > Introduce a Cluster Monitor that aggregates load information from individual > Node Managers and computes an ordered list of preferred Node managers to be > used as target Nodes for OPPORTUNISTIC container allocations. > This list can be pushed out to the Node Manager (specifically the AMRMProxy > running on the Node) via the Allocate Response. This will be used to make > local Scheduling decisions -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-4412) Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers
[ https://issues.apache.org/jira/browse/YARN-4412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-4412: -- Attachment: YARN-4412.addendum-001.patch [~gtCarrera9], Like I mentioned.. I fixed most of the relevant ones in YARN-2888. Some of them like the ones pertaining to TopKNodeSelector are not relevant anymore (the class itself does not exit after YARN-2888) In any case, I am attaching new addendum patch with remaining possible checkstyle fixes.. > Create ClusterMonitor to compute ordered list of preferred NMs for > OPPORTUNITIC containers > -- > > Key: YARN-4412 > URL: https://issues.apache.org/jira/browse/YARN-4412 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Arun Suresh >Assignee: Arun Suresh > Fix For: 3.0.0-alpha1 > > Attachments: YARN-4412-yarn-2877.v1.patch, > YARN-4412-yarn-2877.v2.patch, YARN-4412-yarn-2877.v3.patch, > YARN-4412-yarn-2877.v4.patch, YARN-4412-yarn-2877.v5.patch, > YARN-4412-yarn-2877.v6.patch, YARN-4412.007.patch, YARN-4412.008.patch, > YARN-4412.009.patch, YARN-4412.addendum-001.patch, YARN-4412.find-bugs.patch > > > Introduce a Cluster Monitor that aggregates load information from individual > Node Managers and computes an ordered list of preferred Node managers to be > used as target Nodes for OPPORTUNISTIC container allocations. > This list can be pushed out to the Node Manager (specifically the AMRMProxy > running on the Node) via the Allocate Response. This will be used to make > local Scheduling decisions -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-4412) Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers
[
https://issues.apache.org/jira/browse/YARN-4412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun Suresh updated YARN-4412:
--
Attachment: YARN-4412.find-bugs.patch
Thanks for the pointer [~vinodkv]. Attaching an addendum patch to fix the
{{findbugs-exclude.xml}} file
> Create ClusterMonitor to compute ordered list of preferred NMs for
> OPPORTUNITIC containers
> --
>
> Key: YARN-4412
> URL: https://issues.apache.org/jira/browse/YARN-4412
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Arun Suresh
>Assignee: Arun Suresh
> Fix For: 3.0.0-alpha1
>
> Attachments: YARN-4412-yarn-2877.v1.patch,
> YARN-4412-yarn-2877.v2.patch, YARN-4412-yarn-2877.v3.patch,
> YARN-4412-yarn-2877.v4.patch, YARN-4412-yarn-2877.v5.patch,
> YARN-4412-yarn-2877.v6.patch, YARN-4412.007.patch, YARN-4412.008.patch,
> YARN-4412.009.patch, YARN-4412.find-bugs.patch
>
>
> Introduce a Cluster Monitor that aggregates load information from individual
> Node Managers and computes an ordered list of preferred Node managers to be
> used as target Nodes for OPPORTUNISTIC container allocations.
> This list can be pushed out to the Node Manager (specifically the AMRMProxy
> running on the Node) via the Allocate Response. This will be used to make
> local Scheduling decisions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-4412) Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers
[ https://issues.apache.org/jira/browse/YARN-4412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-4412: -- Attachment: YARN-4412.009.patch > Create ClusterMonitor to compute ordered list of preferred NMs for > OPPORTUNITIC containers > -- > > Key: YARN-4412 > URL: https://issues.apache.org/jira/browse/YARN-4412 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Arun Suresh >Assignee: Arun Suresh > Attachments: YARN-4412-yarn-2877.v1.patch, > YARN-4412-yarn-2877.v2.patch, YARN-4412-yarn-2877.v3.patch, > YARN-4412-yarn-2877.v4.patch, YARN-4412-yarn-2877.v5.patch, > YARN-4412-yarn-2877.v6.patch, YARN-4412.007.patch, YARN-4412.008.patch, > YARN-4412.009.patch > > > Introduce a Cluster Monitor that aggregates load information from individual > Node Managers and computes an ordered list of preferred Node managers to be > used as target Nodes for OPPORTUNISTIC container allocations. > This list can be pushed out to the Node Manager (specifically the AMRMProxy > running on the Node) via the Allocate Response. This will be used to make > local Scheduling decisions -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4412) Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers
[
https://issues.apache.org/jira/browse/YARN-4412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun Suresh updated YARN-4412:
--
Attachment: YARN-4412.008.patch
Attaching patch to fix the {{TestYarnConfiguration}} testcase.
* The Findbugs will not be fixed.. as per this
[comment|https://issues.apache.org/jira/browse/YARN-4412?focusedCommentId=15148978=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15148978]
ping [~kasha], [~curino], [~leftnoteasy], Do let me know if you have any
reservations to cherry-picking this to trunk (given that it has been +1-ed for
the branch)
> Create ClusterMonitor to compute ordered list of preferred NMs for
> OPPORTUNITIC containers
> --
>
> Key: YARN-4412
> URL: https://issues.apache.org/jira/browse/YARN-4412
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Arun Suresh
>Assignee: Arun Suresh
> Attachments: YARN-4412-yarn-2877.v1.patch,
> YARN-4412-yarn-2877.v2.patch, YARN-4412-yarn-2877.v3.patch,
> YARN-4412-yarn-2877.v4.patch, YARN-4412-yarn-2877.v5.patch,
> YARN-4412-yarn-2877.v6.patch, YARN-4412.007.patch, YARN-4412.008.patch
>
>
> Introduce a Cluster Monitor that aggregates load information from individual
> Node Managers and computes an ordered list of preferred Node managers to be
> used as target Nodes for OPPORTUNISTIC container allocations.
> This list can be pushed out to the Node Manager (specifically the AMRMProxy
> running on the Node) via the Allocate Response. This will be used to make
> local Scheduling decisions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-4412) Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers
[ https://issues.apache.org/jira/browse/YARN-4412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-4412: -- Attachment: YARN-4412.007.patch Uploading patch after rebase with trunk.. > Create ClusterMonitor to compute ordered list of preferred NMs for > OPPORTUNITIC containers > -- > > Key: YARN-4412 > URL: https://issues.apache.org/jira/browse/YARN-4412 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Arun Suresh >Assignee: Arun Suresh > Attachments: YARN-4412-yarn-2877.v1.patch, > YARN-4412-yarn-2877.v2.patch, YARN-4412-yarn-2877.v3.patch, > YARN-4412-yarn-2877.v4.patch, YARN-4412-yarn-2877.v5.patch, > YARN-4412-yarn-2877.v6.patch, YARN-4412.007.patch > > > Introduce a Cluster Monitor that aggregates load information from individual > Node Managers and computes an ordered list of preferred Node managers to be > used as target Nodes for OPPORTUNISTIC container allocations. > This list can be pushed out to the Node Manager (specifically the AMRMProxy > running on the Node) via the Allocate Response. This will be used to make > local Scheduling decisions -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4412) Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers
[ https://issues.apache.org/jira/browse/YARN-4412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-4412: -- Attachment: YARN-4412-yarn-2877.v6.patch > Create ClusterMonitor to compute ordered list of preferred NMs for > OPPORTUNITIC containers > -- > > Key: YARN-4412 > URL: https://issues.apache.org/jira/browse/YARN-4412 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Arun Suresh >Assignee: Arun Suresh > Attachments: YARN-4412-yarn-2877.v1.patch, > YARN-4412-yarn-2877.v2.patch, YARN-4412-yarn-2877.v3.patch, > YARN-4412-yarn-2877.v4.patch, YARN-4412-yarn-2877.v5.patch, > YARN-4412-yarn-2877.v6.patch > > > Introduce a Cluster Monitor that aggregates load information from individual > Node Managers and computes an ordered list of preferred Node managers to be > used as target Nodes for OPPORTUNISTIC container allocations. > This list can be pushed out to the Node Manager (specifically the AMRMProxy > running on the Node) via the Allocate Response. This will be used to make > local Scheduling decisions -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4412) Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers
[ https://issues.apache.org/jira/browse/YARN-4412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-4412: -- Attachment: YARN-4412-yarn-2877.v5.patch Updating patch to add some more documentation.. > Create ClusterMonitor to compute ordered list of preferred NMs for > OPPORTUNITIC containers > -- > > Key: YARN-4412 > URL: https://issues.apache.org/jira/browse/YARN-4412 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Arun Suresh >Assignee: Arun Suresh > Attachments: YARN-4412-yarn-2877.v1.patch, > YARN-4412-yarn-2877.v2.patch, YARN-4412-yarn-2877.v3.patch, > YARN-4412-yarn-2877.v4.patch, YARN-4412-yarn-2877.v5.patch > > > Introduce a Cluster Monitor that aggregates load information from individual > Node Managers and computes an ordered list of preferred Node managers to be > used as target Nodes for OPPORTUNISTIC container allocations. > This list can be pushed out to the Node Manager (specifically the AMRMProxy > running on the Node) via the Allocate Response. This will be used to make > local Scheduling decisions -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4412) Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers
[
https://issues.apache.org/jira/browse/YARN-4412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun Suresh updated YARN-4412:
--
Attachment: YARN-4412-yarn-2877.v4.patch
Updating patch to fix testcases.
With regard to the findbugs warnings:
# The {{System.exit(-1)}} in the {{EventDispatcher}} is intentional
# The Synchronization inconsistencies in the Protocol Buffer classes will
remain.. to maintain consistency with all the other PBImpl classes. Also, I do
not expect multiple threads to be modifying the same object in any case.
> Create ClusterMonitor to compute ordered list of preferred NMs for
> OPPORTUNITIC containers
> --
>
> Key: YARN-4412
> URL: https://issues.apache.org/jira/browse/YARN-4412
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Arun Suresh
>Assignee: Arun Suresh
> Attachments: YARN-4412-yarn-2877.v1.patch,
> YARN-4412-yarn-2877.v2.patch, YARN-4412-yarn-2877.v3.patch,
> YARN-4412-yarn-2877.v4.patch
>
>
> Introduce a Cluster Monitor that aggregates load information from individual
> Node Managers and computes an ordered list of preferred Node managers to be
> used as target Nodes for OPPORTUNISTIC container allocations.
> This list can be pushed out to the Node Manager (specifically the AMRMProxy
> running on the Node) via the Allocate Response. This will be used to make
> local Scheduling decisions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-4412) Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers
[ https://issues.apache.org/jira/browse/YARN-4412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-4412: -- Attachment: YARN-4412-yarn-2877.v3.patch Uploading patch to rebase with the latest YARN-2877 as well as address [~curino]'s review comments > Create ClusterMonitor to compute ordered list of preferred NMs for > OPPORTUNITIC containers > -- > > Key: YARN-4412 > URL: https://issues.apache.org/jira/browse/YARN-4412 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Arun Suresh >Assignee: Arun Suresh > Attachments: YARN-4412-yarn-2877.v1.patch, > YARN-4412-yarn-2877.v2.patch, YARN-4412-yarn-2877.v3.patch > > > Introduce a Cluster Monitor that aggregates load information from individual > Node Managers and computes an ordered list of preferred Node managers to be > used as target Nodes for OPPORTUNISTIC container allocations. > This list can be pushed out to the Node Manager (specifically the AMRMProxy > running on the Node) via the Allocate Response. This will be used to make > local Scheduling decisions -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4412) Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers
[ https://issues.apache.org/jira/browse/YARN-4412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantinos Karanasos updated YARN-4412: - Assignee: Arun Suresh (was: Konstantinos Karanasos) > Create ClusterMonitor to compute ordered list of preferred NMs for > OPPORTUNITIC containers > -- > > Key: YARN-4412 > URL: https://issues.apache.org/jira/browse/YARN-4412 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Arun Suresh >Assignee: Arun Suresh > Attachments: YARN-4412-yarn-2877.v1.patch > > > Introduce a Cluster Monitor that aggregates load information from individual > Node Managers and computes an ordered list of preferred Node managers to be > used as target Nodes for OPPORTUNISTIC container allocations. > This list can be pushed out to the Node Manager (specifically the AMRMProxy > running on the Node) via the Allocate Response. This will be used to make > local Scheduling decisions -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4412) Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers
[ https://issues.apache.org/jira/browse/YARN-4412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantinos Karanasos updated YARN-4412: - Summary: Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers (was: Create ClusterManager to compute ordered list of preferred NMs for OPPORTUNITIC containers) > Create ClusterMonitor to compute ordered list of preferred NMs for > OPPORTUNITIC containers > -- > > Key: YARN-4412 > URL: https://issues.apache.org/jira/browse/YARN-4412 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Arun Suresh >Assignee: Arun Suresh > Attachments: YARN-4412-yarn-2877.v1.patch > > > Introduce a Cluster Manager that aggregates Load and Policy information from > individual Node Managers and computes an ordered list of preferred Node > managers to be used as target Nodes for OPPORTUNISTIC container allocations. > This list can be pushed out to the Node Manager (specifically the AMRMProxy > running on the Node) via the Allocate Response. This will be used to make > local Scheduling decisions -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4412) Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers
[ https://issues.apache.org/jira/browse/YARN-4412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantinos Karanasos updated YARN-4412: - Description: Introduce a Cluster Monitor that aggregates load information from individual Node Managers and computes an ordered list of preferred Node managers to be used as target Nodes for OPPORTUNISTIC container allocations. This list can be pushed out to the Node Manager (specifically the AMRMProxy running on the Node) via the Allocate Response. This will be used to make local Scheduling decisions was: Introduce a Cluster Manager that aggregates Load and Policy information from individual Node Managers and computes an ordered list of preferred Node managers to be used as target Nodes for OPPORTUNISTIC container allocations. This list can be pushed out to the Node Manager (specifically the AMRMProxy running on the Node) via the Allocate Response. This will be used to make local Scheduling decisions > Create ClusterMonitor to compute ordered list of preferred NMs for > OPPORTUNITIC containers > -- > > Key: YARN-4412 > URL: https://issues.apache.org/jira/browse/YARN-4412 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Arun Suresh >Assignee: Konstantinos Karanasos > Attachments: YARN-4412-yarn-2877.v1.patch > > > Introduce a Cluster Monitor that aggregates load information from individual > Node Managers and computes an ordered list of preferred Node managers to be > used as target Nodes for OPPORTUNISTIC container allocations. > This list can be pushed out to the Node Manager (specifically the AMRMProxy > running on the Node) via the Allocate Response. This will be used to make > local Scheduling decisions -- This message was sent by Atlassian JIRA (v6.3.4#6332)
