[
https://issues.apache.org/jira/browse/YARN-8320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16489398#comment-16489398
]
Miklos Szegedi commented on YARN-8320:
--------------------------------------
[~cheersyang] / [~yangjiandan], thank you for raising this, this would be a
very useful feature.
Thank you [~leftnoteasy] for the comments.
1) I agree with [~leftnoteasy] about the special considerations regarding
rounding. Because of this it might make sense to use a separate resource type
for this feature. See my other comments regarding this below.
2) I also think like [~leftnoteasy] that users might not need the
RESERVED/SHARED modes. It adds complexity reducing the number of users, who
would use the feature. On the other hand I admit it nicely applies to
cpuset.cpu_exclusive=0/1.
3) I definitely agree with [~leftnoteasy] in the use of resource types. It
might be straightforward to have a cpuset resource type that the AMs can
request and share the cgroups accordingly. This would also make the
configuration more standard. The levels might not even be needed in this case.
If an application does not request cpuset, it is shared, otherwise it is
exclusive. The current suggestion would work but please consider using resource
types.
4) The design lets the AM do a delayed exclusive request directly to the NM
avoiding the RM. I think it would be more robust to request from the RM in the
container launch context and just forward this to the NM. The RM has the chance
to decline or delay the request in this case in the future.
5) [~yangjiandan], how can you make sure a parent cgroup does not interfere
with a cgroup marked as {{cpuset.cpu_exclusive=1}}? What if a system service
wakes up?
6) Let me mention that this feature negatively affects YARN-1011 and
oversubscription. An exclusive CPU with leftover cannot be used by any other
container and remains idle. This reduces overall cluster utilization.
7) Also, latency sensitive applications get exclusive protection but can only
be assigned to their cpuset disallowing bursts to other CPUs when needed. I do
not know how to solve this though.
8) If a cpuset is not exclusive it is considered as a limit by cgroups not a
reserve. The feature uses this as a reserve which practically would mean that
other container cgroups need to be changed and reduced every time a reserved
container starts. Am I correct?
> [Umbrella] Support CPU isolation for latency-sensitive (LS) service
> -------------------------------------------------------------------
>
> Key: YARN-8320
> URL: https://issues.apache.org/jira/browse/YARN-8320
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: nodemanager
> Reporter: Jiandan Yang
> Priority: Major
> Attachments: CPU-isolation-for-latency-sensitive-services-v1.pdf,
> CPU-isolation-for-latency-sensitive-services-v2.pdf, YARN-8320.001.patch
>
>
> Currently NodeManager uses “cpu.cfs_period_us”, “cpu.cfs_quota_us” and
> “cpu.shares” to isolate cpu resource. However,
> * Linux Completely Fair Scheduling (CFS) is a throughput-oriented scheduler;
> no support for differentiated latency
> * Request latency of services running on container may be frequent shake
> when all containers share cpus, and latency-sensitive services can not afford
> in our production environment.
> So we need more fine-grained cpu isolation.
> Here we propose a solution using cgroup cpuset to binds containers to
> different processors, this is inspired by the isolation technique in [Borg
> system|http://schd.ws/hosted_files/lcccna2016/a7/CAT%20@%20Scale.pdf].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
