[
https://issues.apache.org/jira/browse/MAPREDUCE-4334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396261#comment-13396261
]
Andrew Ferguson commented on MAPREDUCE-4334:
--------------------------------------------
ok, putting all of this in the ContainerExecutor is not the way to go, as it
precludes use of secure Hadoop's Linux container-executor.
In my new design, ContainerMonitor will be a pluggable component, just as
ContainerExecutor is now. Then, we can provide a ContainerMonitor which uses
cgroups to control resource usage, rather than the existing ContainerMonitor
(to be renamed as "DefaultContainerMonitor"). This has several advantages:
1) allows us to keep existing ContainerMonitor for users who can't use cgroups
(eg, users without root access during Hadoop setup)
2) ContainerMonitor already receives an event when it's time to stop
monitoring, which we can use as notification to delete the container's cgroup
3) ContainerMonitor receives the resource limits already; no need to calculate
them based on the configs
4) A pluggable ContainerMonitor paves the way for ContainerMonitors on other
platforms
I will first open a sub-task to make ContainerMonitor pluggable.
The only trouble spot with this design is that it's not possible to move
another non-root user's process into a cgroup. I plan to extend the secure
container-executor to be able to make such a move.
Please let me know if you have any feedback about this proposal.
thank you,
Andrew
> Add support for CPU isolation/monitoring of containers
> ------------------------------------------------------
>
> Key: MAPREDUCE-4334
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4334
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Reporter: Arun C Murthy
> Assignee: Arun C Murthy
>
> Once we get in MAPREDUCE-4327, it will be important to actually enforce
> limits on CPU consumption of containers.
> Several options spring to mind:
> # taskset (RHEL5+)
> # cgroups (RHEL6+)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
