[
https://issues.apache.org/jira/browse/YARN-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16102371#comment-16102371
]
Arun Suresh edited comment on YARN-6870 at 7/27/17 2:22 AM:
------------------------------------------------------------
[~brookz], I think we should start with addressing the bug in this jira. We can
probably make a change in the internal protobuf representation if we find it
useful for other downstream features like YARN-1011.
was (Author: asuresh):
@Brook Zhou, I think we should start with addressing the bug in this jira. We
can probably change in the internal protobuf representation if we find it
useful for other downstream features like YARN-1011.
> ResourceUtilization/ContainersMonitorImpl is calculating CPU utilization as a
> float, which is imprecise
> -------------------------------------------------------------------------------------------------------
>
> Key: YARN-6870
> URL: https://issues.apache.org/jira/browse/YARN-6870
> Project: Hadoop YARN
> Issue Type: Bug
> Components: api, nodemanager
> Reporter: Brook Zhou
> Assignee: Brook Zhou
> Attachments: YARN-6870-v0.patch
>
>
> We have seen issues on our clusters where the current way of computing CPU
> usage is having float-arithmetic inaccuracies (the bug is still there in
> trunk)
> Simple program to illustrate:
> {code:title=Bar.java|borderStyle=solid}
> public static void main(String[] args) throws Exception {
> float result = 0.0f;
> for (int i = 0; i < 7; i++) {
> if (i == 6) {
> result += (float) 4 / (float)18;
> } else {
> result += (float) 2 / (float)18;
> }
> }
> for (int i = 0; i < 7; i++) {
> if (i == 6) {
> result -= (float) 4 / (float)18;
> } else {
> result -= (float) 2 / (float)18;
> }
> }
> System.out.println(result);
> }
> {code}
> // Printed
> 4.4703484E-8
> 2017-04-12 05:43:24,014 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl:
> Not enough cpu for [container_e3295_1491978508342_0467_01_000030], Current
> CPU Allocation: [0.8888891], Requested CPU Allocation: [0.11111111]
> There are a few places with this issue:
> 1. ResourceUtilization.java - set/getCPU both use float. When
> ContainerScheduler calls
> ContainersMonitor.increase/decreaseResourceUtilization, this may lead to
> issues.
> 2. AllocationBasedResourceUtilizationTracker.java - hasResourcesAvailable
> uses float as well for CPU computation.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
