[
https://issues.apache.org/jira/browse/YARN-11669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18019215#comment-18019215
]
Mykolas K edited comment on YARN-11669 at 9/9/25 7:25 PM:
----------------------------------------------------------
Hi [~bteke], we're really missing cgroup v2 support in YARN. As the
industry/community is dropping support for cgroup v1, what could we do to help
ship out 3.5.0 and cgroup v2 support in YARN?
If GPU support is the only remaining blocker could we at least release the rest
of cgroup v2 support and get back to GPU support with a smaller patch down the
line? Thanks!
was (Author: JIRAUSER310053):
Hi [~bteke], we're really missing cgroup v2 support in YARN. As the
industry/community is dropping support for cgroup v1, what could we do to ship
out 3.5.0 and cgroup v2 support in YARN?
If GPU support is the only remaining blocker could we at least release the rest
of cgroup v2 support and get back to GPU support with a smaller patch down the
line? Thanks!
> [Umbrella] cgroup v2 support
> ----------------------------
>
> Key: YARN-11669
> URL: https://issues.apache.org/jira/browse/YARN-11669
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: yarn
> Reporter: Ferenc Erdelyi
> Assignee: Benjamin Teke
> Priority: Major
> Fix For: 3.5.0
>
>
> cgroup v2 has some fundamental changes compared to v1. RHEL9, Ubuntu 22
> already moved to cgroup v2 as a default, hence YARN should support it. This
> umbrella tracks the required work.
> [Documentation|https://docs.kernel.org/admin-guide/cgroup-v2.html]
> A way to test the newly added features:
> # Turn on cgroup v1 based on the current
> [documentation|https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeManagerCgroups.html].
> # System prerequisites:
> ## the file {{/etc/mtab}} should contain a mount path with the file system
> type {{cgroup2}}, by default this could be {{/sys/fs/cgroup}} on most OS's
> ## the {{cgroup.subtree_control}} file should contain the necessary
> controllers (update it with: {{echo "+cpu +io +memory" >
> cgroup.subtree_control}})
> ## either create the YARN hierarchy and give recursive access to the user
> running the NM on the node. The hierarchy is {{hadoop-yarn}} by default
> (controller by
> {{yarn.nodemanager.linux-container-executor.cgroups.hierarchy}}), and
> recursive mode is required, because as soon as the directory is created it
> will be filled with the controller files which YARN will try to edit.
> ### Alternatively if the NM process user has access rights on the
> {{/sys/fs/cgroup}} directory it'll try to create the hierarchy and update the
> {{cgroup.subtree_control}} file.
> # YARN configuration
> ## {{yarn.nodemanager.linux-container-executor.cgroups.mount-path}} should
> point to the directory where the cgroup2 structure is mounted and the
> {{hadoop-yarn}} hierarchy was created
> ## {{yarn.nodemanager.linux-container-executor.cgroups.v2.enabled}} should be
> set to {{true}}
> ## Enable a cgroup controller, like {{yarn. nodemanager. resource.
> cpu.enabled}}: {{true}}
> # Launch the NM and monitor the cgroup files on container launches (i.e:
> {{/sys/fs/cgroup/hadoop-yarn/container_id/cpu.weight}})
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
