[
https://issues.apache.org/jira/browse/YARN-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14110555#comment-14110555
]
Varun Vasudev commented on YARN-2440:
-------------------------------------
[~jlowe] the example provided by [~sjlee0] is the one I wanted to address when
I added support for both percentage and absolute cores. Would it make more
sense if I picked the lower value instead of one overriding the other.
Something like -
1. Evaluate cores allocated by yarn.nodemanager.containers-cpu-cores and
yarn.nodemanager.containers-cpu-percentage.
2. Pick the lower of the two values
3. Log a warning/info message that both were specified and that we're picking
the lower value.
{quote}
I'm not thrilled about the name template "containers-cpu-*" since it could
easily be misinterpreted as a per-container thing as well, but I'm currently at
a loss for a better prefix. Suggestions welcome.
{quote}
How about yarn.nodemanager.all-containers-cpu-cores and
yarn.nodemanager.all-containers-cpu-percentage?
{quote}
Does getOverallLimits need to check for a quotaUS that's too low as well?
{quote}
Thanks for catching this; I'll fix it in the next patch.
{quote}
I think minimally we need to log a warning if we're going to ignore setting up
cgroups to limit CPU usage across all containers if the user specified to do so.
{quote}
I'll add in the logging message.
{quote}
Related to the previous comment, I think it would be nice if we didn't try to
setup any limits if none were specified. That way if there's some issue with
correctly determining the number of cores on a particular system it can still
work in the default, "use everything" scenario.
{quote}
Will do.
{quote}
NodeManagerHardwareUtils.getContainerCores should be getContainersCores (the
per-container vs. all-containers confusion again)
{quote}
I'll rename the function.
> Cgroups should allow YARN containers to be limited to allocated cores
> ---------------------------------------------------------------------
>
> Key: YARN-2440
> URL: https://issues.apache.org/jira/browse/YARN-2440
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Varun Vasudev
> Assignee: Varun Vasudev
> Attachments: apache-yarn-2440.0.patch, apache-yarn-2440.1.patch,
> apache-yarn-2440.2.patch, screenshot-current-implementation.jpg
>
>
> The current cgroups implementation does not limit YARN containers to the
> cores allocated in yarn-site.xml.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
