[
https://issues.apache.org/jira/browse/YARN-3119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304170#comment-14304170
]
Anubhav Dhoot commented on YARN-3119:
-------------------------------------
Thanks for the comments.
Just wanted to give a quick response on the configurability. In my prelim patch
I did add the option to turn it off by default though i could use a better name
(DEFAULT_TOTAL_MEM_USAGE_CHECK_ENABLED is set to false). I also added a
configurable threshold that lets you control how much of the overall YARN
container memory is allowed to be exceeded before we trigger the enforcement.
(Setting it to 0 would be another way to revert back to old behavior)
> Memory limit check need not be enforced unless aggregate usage of all
> containers is near limit
> ----------------------------------------------------------------------------------------------
>
> Key: YARN-3119
> URL: https://issues.apache.org/jira/browse/YARN-3119
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager
> Reporter: Anubhav Dhoot
> Assignee: Anubhav Dhoot
> Attachments: YARN-3119.prelim.patch
>
>
> Today we kill any container preemptively even if the total usage of
> containers for that is well within the limit for YARN. Instead if we enforce
> memory limit only if the total limit of all containers is close to some
> configurable ratio of overall memory assigned to containers, we can allow for
> flexibility in container memory usage without adverse effects. This is
> similar in principle to how cgroups uses soft_limit_in_bytes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
