[
https://issues.apache.org/jira/browse/MAPREDUCE-4351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13397794#comment-13397794
]
Andrew Ferguson commented on MAPREDUCE-4351:
--------------------------------------------
Hi Arun,
I initially considered that approach (having ContainersMonitor go away), but
decided against it for the following reasons:
- Because the initial configuration of cgroups requires root access, some users
may not be able to use them. Therefore, it makes sense to keep the current
ContainersMonitor, which monitors memory usage and does its own OOM killing.
- Maintaining the separation between the ContainerExecutor (which just does
launching) and the ContainersMonitor (which just does enforcement) lowers the
amount of code overlap for supporting each permutation: secure executor
with/without cgroups, default executor with/without cgroups, a
ContainersMonitor with taskset for CPU instead of cgroups, etc.
- With this approach, the ContainersMonitor continues to receive the
ContainerStopMonitoringEvent event, which allows it to delete cgroups which are
no longer in use.
- Finally, the current ContainersMonitor does not start enforcing the memory
limit until after a short delay because of the potential to double-count
resources when the JVM does a fork()+exec() -- see comment above
isProcessTreeOverLimit(). Keeping resource enforcement separate from process
launching gives us the flexibility to delay the start of enforcement if needed.
do these reasons make sense?
I'm currently writing a "CgroupsContainersMonitor" to complement the
"DefaultContainersMonitor," and which plugs-in using this patch. As you point
out, this CgroupsContainerMonitor doesn't do much work: it just configures
cgroups and places processes in them, allowing the kernel to do the actual
enforcement work.
thanks!
Andrew
> Make ContainersMonitor pluggable
> --------------------------------
>
> Key: MAPREDUCE-4351
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4351
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, nodemanager
> Reporter: Andrew Ferguson
> Assignee: Andrew Ferguson
> Attachments: MAPREDUCE-4351-v1.patch, MAPREDUCE-4351-v2.patch,
> MAPREDUCE-4351-v3.patch, MAPREDUCE-4351-v4.patch, MAPREDUCE-4351-v4.patch
>
>
> Make the existing ContainersManager pluggable, just as the ContainerExecutor
> is currently. This will allow us to add container resource enforcement using
> other techniques (such as cgroups) in an extensible fashion.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
