[
https://issues.apache.org/jira/browse/YARN-9024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688214#comment-16688214
]
Wilfred Spiegelenburg commented on YARN-9024:
---------------------------------------------
Thank you for logging this issue and providing the a patch.
I have a question based on the logic used: in your fix you add the logic for
converting the unit inside the check if the value is larger than the current
maximum known for the type. This looks counter intuitive. Should we not convert
the units before we check the value? i.e. The current maximum known is 1000
"base". The node registers with 1 "G" as the resource value. It has a smaller
"value" component but after unit conversion to the "base" it is really larger
than the current max.
> ClusterNodeTracker maximum allocation does not respect resource units
> ---------------------------------------------------------------------
>
> Key: YARN-9024
> URL: https://issues.apache.org/jira/browse/YARN-9024
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Szilard Nemeth
> Assignee: Szilard Nemeth
> Priority: Major
> Attachments: YARN-9024.001.patch
>
>
> If a custom resource is defined with a default unit value (base unit) and a
> node reports its total capability in a different unit (e.g. M) then
> {{ClusterNodeTracker.getMaxAllowedAllocation}} returns the max allocation
> resource in the base unit, so the reported resource unit is not respected.
> The issue is when the \{{updateMaxResources}} method is called (i.e. NM node
> is registered), the unit of the node's resources is not checked. In this
> method, we need to convert the reported value to the unit defined by RM for
> the individual resource types.
> I also wanted to add a testcase where memory has G as its unit, but it was
> not possible easily without hacky code so I only added a testcase that
> verifies custom resource values.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
