[jira] [Commented] (YARN-6673) Add cpu cgroup configurations for opportunistic containers
[ https://issues.apache.org/jira/browse/YARN-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16312473#comment-16312473 ] Jiandan Yang commented on YARN-6673: - [~miklos.szeg...@cloudera.com] How about setting Cpu share for Opportunistic container * CPU_DEFAULT_WEIGHT_OPPORTUNISTIC * containerVCores* > Add cpu cgroup configurations for opportunistic containers > -- > > Key: YARN-6673 > URL: https://issues.apache.org/jira/browse/YARN-6673 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Haibo Chen >Assignee: Miklos Szegedi > Fix For: 3.0.0-beta1 > > Attachments: YARN-6673.000.patch > > > In addition to setting cpu.cfs_period_us on a per-container basis, we could > also set cpu.shares to 2 for opportunistic containers so they are run on a > best-effort basis -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6673) Add cpu cgroup configurations for opportunistic containers
[ https://issues.apache.org/jira/browse/YARN-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16312476#comment-16312476 ] Miklos Szegedi commented on YARN-6673: -- [~yangjiandan], would not that allow opportunistic containers to steal CPU cycles from guaranteed ones by setting the vcores to arbitrary size like 512? [~haibochen], is there an upper limit to vcores in RM? > Add cpu cgroup configurations for opportunistic containers > -- > > Key: YARN-6673 > URL: https://issues.apache.org/jira/browse/YARN-6673 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Haibo Chen >Assignee: Miklos Szegedi > Fix For: 3.0.0-beta1 > > Attachments: YARN-6673.000.patch > > > In addition to setting cpu.cfs_period_us on a per-container basis, we could > also set cpu.shares to 2 for opportunistic containers so they are run on a > best-effort basis -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6673) Add cpu cgroup configurations for opportunistic containers
[ https://issues.apache.org/jira/browse/YARN-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16313529#comment-16313529 ] Miklos Szegedi commented on YARN-6673: -- [~yangjiandan], I am assuming the requestor specifies more vcores if it is capable running more threads. We have the following scenarios: 1. There are only guaranteed containers on the node: In this case it does not matter what the opportunistic weight is 2. There are guaranteed containers that use the whole node and some opportunistic ones allocated: The opportunistic weight should be minimal (2), so that opportunistic containers are fully throttled without any other action from the node manager. 3. There are guaranteed containers that leave a gap and some opportunistic ones allocated: a) Let's say there are 3 physical cores left and there are two opportunistic containers with 2 and 1 threads each allocating 2 and 1 vcores respectively. Each thread gets a physical core, so the remaining resource is shared like (2,1) b) There is just one core left and there are two opportunistic containers with 2 and 1 threads each allocating 2 and 1 vcores respectively. Each thread gets a weight of 2, so the remaining CPU should be shared like (4,2)=(2,1). 4. There are only opportunistic containers: They will be promoted to guaranteed and the standard logic applies. > Add cpu cgroup configurations for opportunistic containers > -- > > Key: YARN-6673 > URL: https://issues.apache.org/jira/browse/YARN-6673 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Haibo Chen >Assignee: Miklos Szegedi > Fix For: 3.0.0-beta1 > > Attachments: YARN-6673.000.patch > > > In addition to setting cpu.cfs_period_us on a per-container basis, we could > also set cpu.shares to 2 for opportunistic containers so they are run on a > best-effort basis -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6673) Add cpu cgroup configurations for opportunistic containers
[ https://issues.apache.org/jira/browse/YARN-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049836#comment-16049836 ] ASF GitHub Bot commented on YARN-6673: -- GitHub user szegedim opened a pull request: https://github.com/apache/hadoop/pull/232 YARN-6673 Add cpu cgroup configurations for opportunistic containers YARN-6673 Add cpu cgroup configurations for opportunistic containers You can merge this pull request into a Git repository by running: $ git pull https://github.com/szegedim/hadoop YARN-6673 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hadoop/pull/232.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #232 commit 51900cb30aad1af61499f85ab2f9b670e5635b82 Author: Miklos Szegedi Date: 2017-06-15T00:06:36Z YARN-6673 Add cpu cgroup configurations for opportunistic containers > Add cpu cgroup configurations for opportunistic containers > -- > > Key: YARN-6673 > URL: https://issues.apache.org/jira/browse/YARN-6673 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Haibo Chen >Assignee: Miklos Szegedi > > In addition to setting cpu.cfs_period_us on a per-container basis, we could > also set cpu.shares to 2 for opportunistic containers so they are run on a > best-effort basis -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6673) Add cpu cgroup configurations for opportunistic containers
[
https://issues.apache.org/jira/browse/YARN-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16056094#comment-16056094
]
ASF GitHub Bot commented on YARN-6673:
--
Github user haibchen commented on a diff in the pull request:
https://github.com/apache/hadoop/pull/232#discussion_r123036136
--- Diff:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsCpuResourceHandlerImpl.java
---
@@ -294,4 +296,26 @@ public void testTeardown() throws Exception {
public void testStrictResourceUsage() throws Exception {
Assert.assertNull(cGroupsCpuResourceHandler.teardown());
}
+
+ @Test
+ public void testOpportunistic() throws Exception {
+Configuration conf = new YarnConfiguration();
+
+String id = "container_01_01";
+ContainerId mockContainerId = mock(ContainerId.class);
+when(mockContainerId.toString()).thenReturn(id);
+
+cGroupsCpuResourceHandler.bootstrap(plugin, conf);
--- End diff --
Can we move this above cGroupsCpuResourceHandler.prestart()?
> Add cpu cgroup configurations for opportunistic containers
> --
>
> Key: YARN-6673
> URL: https://issues.apache.org/jira/browse/YARN-6673
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Haibo Chen
>Assignee: Miklos Szegedi
>
> In addition to setting cpu.cfs_period_us on a per-container basis, we could
> also set cpu.shares to 2 for opportunistic containers so they are run on a
> best-effort basis
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6673) Add cpu cgroup configurations for opportunistic containers
[
https://issues.apache.org/jira/browse/YARN-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16056096#comment-16056096
]
ASF GitHub Bot commented on YARN-6673:
--
Github user haibchen commented on a diff in the pull request:
https://github.com/apache/hadoop/pull/232#discussion_r123031306
--- Diff:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsCpuResourceHandlerImpl.java
---
@@ -181,16 +184,23 @@ public static boolean cpuLimitsExist(String path)
@Override
public List preStart(Container container)
throws ResourceHandlerException {
-
String cgroupId = container.getContainerId().toString();
Resource containerResource = container.getResource();
cGroupsHandler.createCGroup(CPU, cgroupId);
try {
int containerVCores = containerResource.getVirtualCores();
- int cpuShares = CPU_DEFAULT_WEIGHT * containerVCores;
- cGroupsHandler
- .updateCGroupParam(CPU, cgroupId,
CGroupsHandler.CGROUP_CPU_SHARES,
- String.valueOf(cpuShares));
+ ContainerTokenIdentifier id =
container.getContainerTokenIdentifier();
+ if (id != null && id.getExecutionType() ==
+ ExecutionType.OPPORTUNISTIC) {
--- End diff --
We can have a small method for reuse I think
boolean isOpportunistic(Container container) {
ContainerTokenIdentifier id = container.getContainerTokenIdentifier();
return id !=null && id.getExecutionType() ==
ExecutionType.OPPORTUNISTIC;
}
> Add cpu cgroup configurations for opportunistic containers
> --
>
> Key: YARN-6673
> URL: https://issues.apache.org/jira/browse/YARN-6673
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Haibo Chen
>Assignee: Miklos Szegedi
>
> In addition to setting cpu.cfs_period_us on a per-container basis, we could
> also set cpu.shares to 2 for opportunistic containers so they are run on a
> best-effort basis
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6673) Add cpu cgroup configurations for opportunistic containers
[ https://issues.apache.org/jira/browse/YARN-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16056095#comment-16056095 ] ASF GitHub Bot commented on YARN-6673: -- Github user haibchen commented on a diff in the pull request: https://github.com/apache/hadoop/pull/232#discussion_r123030657 --- Diff: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsCpuResourceHandlerImpl.java --- @@ -72,6 +74,7 @@ static final int MIN_PERIOD_US = 1000; @VisibleForTesting static final int CPU_DEFAULT_WEIGHT = 1024; // set by kernel --- End diff -- Can we rename CPU_DEFAULT_WEIGHT to CPU_DEFAULT_WEIGHT_GUARANTEED for clarity? > Add cpu cgroup configurations for opportunistic containers > -- > > Key: YARN-6673 > URL: https://issues.apache.org/jira/browse/YARN-6673 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Haibo Chen >Assignee: Miklos Szegedi > > In addition to setting cpu.cfs_period_us on a per-container basis, we could > also set cpu.shares to 2 for opportunistic containers so they are run on a > best-effort basis -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6673) Add cpu cgroup configurations for opportunistic containers
[
https://issues.apache.org/jira/browse/YARN-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16056093#comment-16056093
]
ASF GitHub Bot commented on YARN-6673:
--
Github user haibchen commented on a diff in the pull request:
https://github.com/apache/hadoop/pull/232#discussion_r123036016
--- Diff:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsCpuResourceHandlerImpl.java
---
@@ -181,16 +184,23 @@ public static boolean cpuLimitsExist(String path)
@Override
public List preStart(Container container)
throws ResourceHandlerException {
-
String cgroupId = container.getContainerId().toString();
Resource containerResource = container.getResource();
cGroupsHandler.createCGroup(CPU, cgroupId);
try {
int containerVCores = containerResource.getVirtualCores();
- int cpuShares = CPU_DEFAULT_WEIGHT * containerVCores;
- cGroupsHandler
- .updateCGroupParam(CPU, cgroupId,
CGroupsHandler.CGROUP_CPU_SHARES,
- String.valueOf(cpuShares));
+ ContainerTokenIdentifier id =
container.getContainerTokenIdentifier();
+ if (id != null && id.getExecutionType() ==
+ ExecutionType.OPPORTUNISTIC) {
+cGroupsHandler
+.updateCGroupParam(CPU, cgroupId,
CGroupsHandler.CGROUP_CPU_SHARES,
+String.valueOf(CPU_DEFAULT_WEIGHT_OPPORTUNISTIC));
+ } else {
+int cpuShares = CPU_DEFAULT_WEIGHT * containerVCores;
+cGroupsHandler
+.updateCGroupParam(CPU, cgroupId,
CGroupsHandler.CGROUP_CPU_SHARES,
+String.valueOf(cpuShares));
+ }
if (strictResourceUsageMode) {
--- End diff --
If strict resource usage mode is turned on, do we need to do differently
for OPPORTUNISTIC containers? Now it bases the cfs_ configurations on vcore
> Add cpu cgroup configurations for opportunistic containers
> --
>
> Key: YARN-6673
> URL: https://issues.apache.org/jira/browse/YARN-6673
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Haibo Chen
>Assignee: Miklos Szegedi
>
> In addition to setting cpu.cfs_period_us on a per-container basis, we could
> also set cpu.shares to 2 for opportunistic containers so they are run on a
> best-effort basis
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6673) Add cpu cgroup configurations for opportunistic containers
[
https://issues.apache.org/jira/browse/YARN-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16056420#comment-16056420
]
ASF GitHub Bot commented on YARN-6673:
--
Github user szegedim commented on a diff in the pull request:
https://github.com/apache/hadoop/pull/232#discussion_r123089455
--- Diff:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsCpuResourceHandlerImpl.java
---
@@ -181,16 +184,23 @@ public static boolean cpuLimitsExist(String path)
@Override
public List preStart(Container container)
throws ResourceHandlerException {
-
String cgroupId = container.getContainerId().toString();
Resource containerResource = container.getResource();
cGroupsHandler.createCGroup(CPU, cgroupId);
try {
int containerVCores = containerResource.getVirtualCores();
- int cpuShares = CPU_DEFAULT_WEIGHT * containerVCores;
- cGroupsHandler
- .updateCGroupParam(CPU, cgroupId,
CGroupsHandler.CGROUP_CPU_SHARES,
- String.valueOf(cpuShares));
+ ContainerTokenIdentifier id =
container.getContainerTokenIdentifier();
+ if (id != null && id.getExecutionType() ==
+ ExecutionType.OPPORTUNISTIC) {
+cGroupsHandler
+.updateCGroupParam(CPU, cgroupId,
CGroupsHandler.CGROUP_CPU_SHARES,
+String.valueOf(CPU_DEFAULT_WEIGHT_OPPORTUNISTIC));
+ } else {
+int cpuShares = CPU_DEFAULT_WEIGHT * containerVCores;
+cGroupsHandler
+.updateCGroupParam(CPU, cgroupId,
CGroupsHandler.CGROUP_CPU_SHARES,
+String.valueOf(cpuShares));
+ }
if (strictResourceUsageMode) {
--- End diff --
They still have a CPU setting, right? Or is it 0?
> Add cpu cgroup configurations for opportunistic containers
> --
>
> Key: YARN-6673
> URL: https://issues.apache.org/jira/browse/YARN-6673
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Haibo Chen
>Assignee: Miklos Szegedi
>
> In addition to setting cpu.cfs_period_us on a per-container basis, we could
> also set cpu.shares to 2 for opportunistic containers so they are run on a
> best-effort basis
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6673) Add cpu cgroup configurations for opportunistic containers
[
https://issues.apache.org/jira/browse/YARN-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16056467#comment-16056467
]
ASF GitHub Bot commented on YARN-6673:
--
Github user haibchen commented on a diff in the pull request:
https://github.com/apache/hadoop/pull/232#discussion_r123096401
--- Diff:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsCpuResourceHandlerImpl.java
---
@@ -181,16 +184,23 @@ public static boolean cpuLimitsExist(String path)
@Override
public List preStart(Container container)
throws ResourceHandlerException {
-
String cgroupId = container.getContainerId().toString();
Resource containerResource = container.getResource();
cGroupsHandler.createCGroup(CPU, cgroupId);
try {
int containerVCores = containerResource.getVirtualCores();
- int cpuShares = CPU_DEFAULT_WEIGHT * containerVCores;
- cGroupsHandler
- .updateCGroupParam(CPU, cgroupId,
CGroupsHandler.CGROUP_CPU_SHARES,
- String.valueOf(cpuShares));
+ ContainerTokenIdentifier id =
container.getContainerTokenIdentifier();
+ if (id != null && id.getExecutionType() ==
+ ExecutionType.OPPORTUNISTIC) {
+cGroupsHandler
+.updateCGroupParam(CPU, cgroupId,
CGroupsHandler.CGROUP_CPU_SHARES,
+String.valueOf(CPU_DEFAULT_WEIGHT_OPPORTUNISTIC));
+ } else {
+int cpuShares = CPU_DEFAULT_WEIGHT * containerVCores;
+cGroupsHandler
+.updateCGroupParam(CPU, cgroupId,
CGroupsHandler.CGROUP_CPU_SHARES,
+String.valueOf(cpuShares));
+ }
if (strictResourceUsageMode) {
--- End diff --
They are currently based on vcore
> Add cpu cgroup configurations for opportunistic containers
> --
>
> Key: YARN-6673
> URL: https://issues.apache.org/jira/browse/YARN-6673
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Haibo Chen
>Assignee: Miklos Szegedi
>
> In addition to setting cpu.cfs_period_us on a per-container basis, we could
> also set cpu.shares to 2 for opportunistic containers so they are run on a
> best-effort basis
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6673) Add cpu cgroup configurations for opportunistic containers
[
https://issues.apache.org/jira/browse/YARN-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16056526#comment-16056526
]
ASF GitHub Bot commented on YARN-6673:
--
Github user szegedim commented on a diff in the pull request:
https://github.com/apache/hadoop/pull/232#discussion_r123105470
--- Diff:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsCpuResourceHandlerImpl.java
---
@@ -181,16 +184,23 @@ public static boolean cpuLimitsExist(String path)
@Override
public List preStart(Container container)
throws ResourceHandlerException {
-
String cgroupId = container.getContainerId().toString();
Resource containerResource = container.getResource();
cGroupsHandler.createCGroup(CPU, cgroupId);
try {
int containerVCores = containerResource.getVirtualCores();
- int cpuShares = CPU_DEFAULT_WEIGHT * containerVCores;
- cGroupsHandler
- .updateCGroupParam(CPU, cgroupId,
CGroupsHandler.CGROUP_CPU_SHARES,
- String.valueOf(cpuShares));
+ ContainerTokenIdentifier id =
container.getContainerTokenIdentifier();
+ if (id != null && id.getExecutionType() ==
+ ExecutionType.OPPORTUNISTIC) {
+cGroupsHandler
+.updateCGroupParam(CPU, cgroupId,
CGroupsHandler.CGROUP_CPU_SHARES,
+String.valueOf(CPU_DEFAULT_WEIGHT_OPPORTUNISTIC));
+ } else {
+int cpuShares = CPU_DEFAULT_WEIGHT * containerVCores;
+cGroupsHandler
+.updateCGroupParam(CPU, cgroupId,
CGroupsHandler.CGROUP_CPU_SHARES,
+String.valueOf(cpuShares));
+ }
if (strictResourceUsageMode) {
--- End diff --
That is a maximum setting and it still needs to be set I think. Why would
opportunistic containers have more resource than guaranteed one?
> Add cpu cgroup configurations for opportunistic containers
> --
>
> Key: YARN-6673
> URL: https://issues.apache.org/jira/browse/YARN-6673
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Haibo Chen
>Assignee: Miklos Szegedi
>
> In addition to setting cpu.cfs_period_us on a per-container basis, we could
> also set cpu.shares to 2 for opportunistic containers so they are run on a
> best-effort basis
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6673) Add cpu cgroup configurations for opportunistic containers
[
https://issues.apache.org/jira/browse/YARN-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16056614#comment-16056614
]
ASF GitHub Bot commented on YARN-6673:
--
Github user haibchen commented on a diff in the pull request:
https://github.com/apache/hadoop/pull/232#discussion_r123117542
--- Diff:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsCpuResourceHandlerImpl.java
---
@@ -181,16 +184,23 @@ public static boolean cpuLimitsExist(String path)
@Override
public List preStart(Container container)
throws ResourceHandlerException {
-
String cgroupId = container.getContainerId().toString();
Resource containerResource = container.getResource();
cGroupsHandler.createCGroup(CPU, cgroupId);
try {
int containerVCores = containerResource.getVirtualCores();
- int cpuShares = CPU_DEFAULT_WEIGHT * containerVCores;
- cGroupsHandler
- .updateCGroupParam(CPU, cgroupId,
CGroupsHandler.CGROUP_CPU_SHARES,
- String.valueOf(cpuShares));
+ ContainerTokenIdentifier id =
container.getContainerTokenIdentifier();
+ if (id != null && id.getExecutionType() ==
+ ExecutionType.OPPORTUNISTIC) {
+cGroupsHandler
+.updateCGroupParam(CPU, cgroupId,
CGroupsHandler.CGROUP_CPU_SHARES,
+String.valueOf(CPU_DEFAULT_WEIGHT_OPPORTUNISTIC));
+ } else {
+int cpuShares = CPU_DEFAULT_WEIGHT * containerVCores;
+cGroupsHandler
+.updateCGroupParam(CPU, cgroupId,
CGroupsHandler.CGROUP_CPU_SHARES,
+String.valueOf(cpuShares));
+ }
if (strictResourceUsageMode) {
--- End diff --
Not quite get your question. What I meant initially was to ask you if you
think base cfs-quota on vcore is the right thing to do.
> Add cpu cgroup configurations for opportunistic containers
> --
>
> Key: YARN-6673
> URL: https://issues.apache.org/jira/browse/YARN-6673
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Haibo Chen
>Assignee: Miklos Szegedi
>
> In addition to setting cpu.cfs_period_us on a per-container basis, we could
> also set cpu.shares to 2 for opportunistic containers so they are run on a
> best-effort basis
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6673) Add cpu cgroup configurations for opportunistic containers
[
https://issues.apache.org/jira/browse/YARN-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16059769#comment-16059769
]
ASF GitHub Bot commented on YARN-6673:
--
Github user szegedim commented on a diff in the pull request:
https://github.com/apache/hadoop/pull/232#discussion_r123582374
--- Diff:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsCpuResourceHandlerImpl.java
---
@@ -181,16 +184,23 @@ public static boolean cpuLimitsExist(String path)
@Override
public List preStart(Container container)
throws ResourceHandlerException {
-
String cgroupId = container.getContainerId().toString();
Resource containerResource = container.getResource();
cGroupsHandler.createCGroup(CPU, cgroupId);
try {
int containerVCores = containerResource.getVirtualCores();
- int cpuShares = CPU_DEFAULT_WEIGHT * containerVCores;
- cGroupsHandler
- .updateCGroupParam(CPU, cgroupId,
CGroupsHandler.CGROUP_CPU_SHARES,
- String.valueOf(cpuShares));
+ ContainerTokenIdentifier id =
container.getContainerTokenIdentifier();
+ if (id != null && id.getExecutionType() ==
+ ExecutionType.OPPORTUNISTIC) {
--- End diff --
Hmm this is just 2 lines of code. Adding a function would be at least 5
more lines. Ideally the container has this function but that might be too much
churn for the interface.
> Add cpu cgroup configurations for opportunistic containers
> --
>
> Key: YARN-6673
> URL: https://issues.apache.org/jira/browse/YARN-6673
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Haibo Chen
>Assignee: Miklos Szegedi
>
> In addition to setting cpu.cfs_period_us on a per-container basis, we could
> also set cpu.shares to 2 for opportunistic containers so they are run on a
> best-effort basis
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6673) Add cpu cgroup configurations for opportunistic containers
[
https://issues.apache.org/jira/browse/YARN-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16059787#comment-16059787
]
ASF GitHub Bot commented on YARN-6673:
--
Github user szegedim commented on a diff in the pull request:
https://github.com/apache/hadoop/pull/232#discussion_r123583294
--- Diff:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsCpuResourceHandlerImpl.java
---
@@ -294,4 +296,26 @@ public void testTeardown() throws Exception {
public void testStrictResourceUsage() throws Exception {
Assert.assertNull(cGroupsCpuResourceHandler.teardown());
}
+
+ @Test
+ public void testOpportunistic() throws Exception {
+Configuration conf = new YarnConfiguration();
+
+String id = "container_01_01";
+ContainerId mockContainerId = mock(ContainerId.class);
+when(mockContainerId.toString()).thenReturn(id);
+
+cGroupsCpuResourceHandler.bootstrap(plugin, conf);
--- End diff --
OKay.
> Add cpu cgroup configurations for opportunistic containers
> --
>
> Key: YARN-6673
> URL: https://issues.apache.org/jira/browse/YARN-6673
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Haibo Chen
>Assignee: Miklos Szegedi
>
> In addition to setting cpu.cfs_period_us on a per-container basis, we could
> also set cpu.shares to 2 for opportunistic containers so they are run on a
> best-effort basis
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6673) Add cpu cgroup configurations for opportunistic containers
[
https://issues.apache.org/jira/browse/YARN-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16059785#comment-16059785
]
ASF GitHub Bot commented on YARN-6673:
--
Github user szegedim commented on a diff in the pull request:
https://github.com/apache/hadoop/pull/232#discussion_r123583254
--- Diff:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsCpuResourceHandlerImpl.java
---
@@ -181,16 +184,23 @@ public static boolean cpuLimitsExist(String path)
@Override
public List preStart(Container container)
throws ResourceHandlerException {
-
String cgroupId = container.getContainerId().toString();
Resource containerResource = container.getResource();
cGroupsHandler.createCGroup(CPU, cgroupId);
try {
int containerVCores = containerResource.getVirtualCores();
- int cpuShares = CPU_DEFAULT_WEIGHT * containerVCores;
- cGroupsHandler
- .updateCGroupParam(CPU, cgroupId,
CGroupsHandler.CGROUP_CPU_SHARES,
- String.valueOf(cpuShares));
+ ContainerTokenIdentifier id =
container.getContainerTokenIdentifier();
+ if (id != null && id.getExecutionType() ==
+ ExecutionType.OPPORTUNISTIC) {
+cGroupsHandler
+.updateCGroupParam(CPU, cgroupId,
CGroupsHandler.CGROUP_CPU_SHARES,
+String.valueOf(CPU_DEFAULT_WEIGHT_OPPORTUNISTIC));
+ } else {
+int cpuShares = CPU_DEFAULT_WEIGHT * containerVCores;
+cGroupsHandler
+.updateCGroupParam(CPU, cgroupId,
CGroupsHandler.CGROUP_CPU_SHARES,
+String.valueOf(cpuShares));
+ }
if (strictResourceUsageMode) {
--- End diff --
Yes, I think so. If the admin chooses strict cpu limits, all containers
should get strict cpu limits based on vcores. Opportunistic ones still will be
throttled by cpu.shares, if guaranteed are running. This is just a cap, for
opportunistic containers with different thread counts not to affect each other
negatively.
> Add cpu cgroup configurations for opportunistic containers
> --
>
> Key: YARN-6673
> URL: https://issues.apache.org/jira/browse/YARN-6673
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Haibo Chen
>Assignee: Miklos Szegedi
>
> In addition to setting cpu.cfs_period_us on a per-container basis, we could
> also set cpu.shares to 2 for opportunistic containers so they are run on a
> best-effort basis
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6673) Add cpu cgroup configurations for opportunistic containers
[
https://issues.apache.org/jira/browse/YARN-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16059881#comment-16059881
]
ASF GitHub Bot commented on YARN-6673:
--
Github user haibchen commented on a diff in the pull request:
https://github.com/apache/hadoop/pull/232#discussion_r123601076
--- Diff:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsCpuResourceHandlerImpl.java
---
@@ -181,16 +184,23 @@ public static boolean cpuLimitsExist(String path)
@Override
public List preStart(Container container)
throws ResourceHandlerException {
-
String cgroupId = container.getContainerId().toString();
Resource containerResource = container.getResource();
cGroupsHandler.createCGroup(CPU, cgroupId);
try {
int containerVCores = containerResource.getVirtualCores();
- int cpuShares = CPU_DEFAULT_WEIGHT * containerVCores;
- cGroupsHandler
- .updateCGroupParam(CPU, cgroupId,
CGroupsHandler.CGROUP_CPU_SHARES,
- String.valueOf(cpuShares));
+ ContainerTokenIdentifier id =
container.getContainerTokenIdentifier();
+ if (id != null && id.getExecutionType() ==
+ ExecutionType.OPPORTUNISTIC) {
--- End diff --
yeah. In some sense, it is to be more descriptive than code reuse. Maybe we
could have an inline boolean variable with such name?
> Add cpu cgroup configurations for opportunistic containers
> --
>
> Key: YARN-6673
> URL: https://issues.apache.org/jira/browse/YARN-6673
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Haibo Chen
>Assignee: Miklos Szegedi
>
> In addition to setting cpu.cfs_period_us on a per-container basis, we could
> also set cpu.shares to 2 for opportunistic containers so they are run on a
> best-effort basis
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6673) Add cpu cgroup configurations for opportunistic containers
[
https://issues.apache.org/jira/browse/YARN-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16059883#comment-16059883
]
ASF GitHub Bot commented on YARN-6673:
--
Github user haibchen commented on a diff in the pull request:
https://github.com/apache/hadoop/pull/232#discussion_r123601141
--- Diff:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsCpuResourceHandlerImpl.java
---
@@ -181,16 +184,23 @@ public static boolean cpuLimitsExist(String path)
@Override
public List preStart(Container container)
throws ResourceHandlerException {
-
String cgroupId = container.getContainerId().toString();
Resource containerResource = container.getResource();
cGroupsHandler.createCGroup(CPU, cgroupId);
try {
int containerVCores = containerResource.getVirtualCores();
- int cpuShares = CPU_DEFAULT_WEIGHT * containerVCores;
- cGroupsHandler
- .updateCGroupParam(CPU, cgroupId,
CGroupsHandler.CGROUP_CPU_SHARES,
- String.valueOf(cpuShares));
+ ContainerTokenIdentifier id =
container.getContainerTokenIdentifier();
+ if (id != null && id.getExecutionType() ==
+ ExecutionType.OPPORTUNISTIC) {
+cGroupsHandler
+.updateCGroupParam(CPU, cgroupId,
CGroupsHandler.CGROUP_CPU_SHARES,
+String.valueOf(CPU_DEFAULT_WEIGHT_OPPORTUNISTIC));
+ } else {
+int cpuShares = CPU_DEFAULT_WEIGHT * containerVCores;
+cGroupsHandler
+.updateCGroupParam(CPU, cgroupId,
CGroupsHandler.CGROUP_CPU_SHARES,
+String.valueOf(cpuShares));
+ }
if (strictResourceUsageMode) {
--- End diff --
I see. Thanks for the clarification.
> Add cpu cgroup configurations for opportunistic containers
> --
>
> Key: YARN-6673
> URL: https://issues.apache.org/jira/browse/YARN-6673
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Haibo Chen
>Assignee: Miklos Szegedi
>
> In addition to setting cpu.cfs_period_us on a per-container basis, we could
> also set cpu.shares to 2 for opportunistic containers so they are run on a
> best-effort basis
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6673) Add cpu cgroup configurations for opportunistic containers
[
https://issues.apache.org/jira/browse/YARN-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16061284#comment-16061284
]
Hadoop QA commented on YARN-6673:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
4s{color} | {color:green} YARN-1011 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} YARN-1011 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} YARN-1011 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
27s{color} | {color:green} YARN-1011 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
43s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
in YARN-1011 has 5 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} YARN-1011 passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 14m
28s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 35m 27s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6673 |
| GITHUB PR | https://github.com/apache/hadoop/pull/232 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 40fbe21f34dd 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | YARN-1011 / 153498b |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-YARN-Build/16233/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/16233/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/16233/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Add cpu cgroup configurations for opportunistic containers
> --
>
> Key: YARN-6673
> URL: https://issues.apache.org/jira/browse/YARN-6673
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Haibo Chen
>Assignee: Miklos Szegedi
> Attachments: YARN-6673.000.patch
>
>
> In addition to setting cpu.cfs_period_us on a per-container bas
[jira] [Commented] (YARN-6673) Add cpu cgroup configurations for opportunistic containers
[ https://issues.apache.org/jira/browse/YARN-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111804#comment-16111804 ] Haibo Chen commented on YARN-6673: -- Thanks [~miklos.szeg...@cloudera.com] for updating the patch. One more knit on the patch. Can we replace the hardcoded expected value in the newly added testOpportunistic() test with references to the variable in CGroupsMemoryResourceHandlerImpl? > Add cpu cgroup configurations for opportunistic containers > -- > > Key: YARN-6673 > URL: https://issues.apache.org/jira/browse/YARN-6673 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Haibo Chen >Assignee: Miklos Szegedi > Attachments: YARN-6673.000.patch > > > In addition to setting cpu.cfs_period_us on a per-container basis, we could > also set cpu.shares to 2 for opportunistic containers so they are run on a > best-effort basis -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6673) Add cpu cgroup configurations for opportunistic containers
[ https://issues.apache.org/jira/browse/YARN-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111848#comment-16111848 ] Miklos Szegedi commented on YARN-6673: -- [~haibochen], I intentionally put the hardcoded value there. If someone changes the value in the production code, it will trigger a test failure. It is just yet another protection to maintain compatibility. > Add cpu cgroup configurations for opportunistic containers > -- > > Key: YARN-6673 > URL: https://issues.apache.org/jira/browse/YARN-6673 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Haibo Chen >Assignee: Miklos Szegedi > Attachments: YARN-6673.000.patch > > > In addition to setting cpu.cfs_period_us on a per-container basis, we could > also set cpu.shares to 2 for opportunistic containers so they are run on a > best-effort basis -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6673) Add cpu cgroup configurations for opportunistic containers
[ https://issues.apache.org/jira/browse/YARN-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111889#comment-16111889 ] Haibo Chen commented on YARN-6673: -- Makes a lot of sense. Will wait until tomorrow morning to check in this in case there is further comments or concerns. > Add cpu cgroup configurations for opportunistic containers > -- > > Key: YARN-6673 > URL: https://issues.apache.org/jira/browse/YARN-6673 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Haibo Chen >Assignee: Miklos Szegedi > Attachments: YARN-6673.000.patch > > > In addition to setting cpu.cfs_period_us on a per-container basis, we could > also set cpu.shares to 2 for opportunistic containers so they are run on a > best-effort basis -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6673) Add cpu cgroup configurations for opportunistic containers
[ https://issues.apache.org/jira/browse/YARN-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16113179#comment-16113179 ] Hudson commented on YARN-6673: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12110 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/12110/]) YARN-6673 Add cpu cgroup configurations for opportunistic containers. (haibochen: rev 293c74a81b407274621751e2104738b2a09e666e) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsCpuResourceHandlerImpl.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsCpuResourceHandlerImpl.java > Add cpu cgroup configurations for opportunistic containers > -- > > Key: YARN-6673 > URL: https://issues.apache.org/jira/browse/YARN-6673 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Haibo Chen >Assignee: Miklos Szegedi > Fix For: 3.0.0-beta1 > > Attachments: YARN-6673.000.patch > > > In addition to setting cpu.cfs_period_us on a per-container basis, we could > also set cpu.shares to 2 for opportunistic containers so they are run on a > best-effort basis -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6673) Add cpu cgroup configurations for opportunistic containers
[ https://issues.apache.org/jira/browse/YARN-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16385440#comment-16385440 ] ASF GitHub Bot commented on YARN-6673: -- Github user szegedim closed the pull request at: https://github.com/apache/hadoop/pull/232 > Add cpu cgroup configurations for opportunistic containers > -- > > Key: YARN-6673 > URL: https://issues.apache.org/jira/browse/YARN-6673 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Haibo Chen >Assignee: Miklos Szegedi >Priority: Major > Fix For: 3.0.0-beta1 > > Attachments: YARN-6673.000.patch > > > In addition to setting cpu.cfs_period_us on a per-container basis, we could > also set cpu.shares to 2 for opportunistic containers so they are run on a > best-effort basis -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
