Zhankun Tang created YARN-8924: ---------------------------------- Summary: Refine the document or code related to legacy CPU isolation/enforcement Key: YARN-8924 URL: https://issues.apache.org/jira/browse/YARN-8924 Project: Hadoop YARN Issue Type: Improvement Reporter: Zhankun Tang Assignee: Zhankun Tang
This is to re-think the legacy configuration/code of CPU resource isolation. In YARN-3542, we involve _CGroupsCpuResourceHandlerImpl_ based on new _ResourceHandler_ mechanism but leaves the configuration "yarn.nodemanager.linux-container-executor.resources-handler.class" there for a long time. Now it seems confusing to the end user. Check YARN-6729, one sets "_DefaultLCEResourcesHandler_" and found that give "percentage-physical-cpu-limit" a value less than "100" doesn't work. As far as I know, internally, the _CgroupsLCEResourcesHandler_ and _DefaultLCEResourcesHandler are_ both deprecated. YARN won't use them anymore. Instead, YARN uses _CGroupsCpuResourceHandlerImpl_ to do CPU isolation and only in LCE. If we want to enforce CPU usage, we must set LCE and CgroupsLCEResourceHandler like this: {noformat} <property> <description>who will execute(launch) the containers.</description> <name>yarn.nodemanager.container-executor.class</name> <value>org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor</value> </property> <property> <description>The class which should help the LCE handle resources.</description> <name>yarn.nodemanager.linux-container-executor.resources-handler.class</name> <value>org.apache.hadoop.yarn.server.nodemanager.util.CgroupsLCEResourcesHandler<value> </property>{noformat} Based on above settings can the CPU related settings like "percentage-physical-cpu-limit" works as expected. To avoid confusing like YARN-6729, we can do two things: # More clear document about how should user configure CPU isolation/enforcement in "NodeManagerCgroups.md" # Make "ResourceHandlerModuler" stable and remove legacy code. Thoughts? [~leftnoteasy], [~vinodkv] -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org