[
https://issues.apache.org/jira/browse/YARN-689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13676164#comment-13676164
]
Alejandro Abdelnur commented on YARN-689:
-----------------------------------------
[~bikassaha], the {{MRAppMaster}} obtains the MIN/MAX values from the
registration response, see the {{RMCommunicator#register()}} method.
And, in the {{RMContainerAllocator#handleEvent()}}} the resource is being
normalized, for example for Map tasks:
{code}
int minSlotMemSize = getMinContainerCapability().getMemory();
mapResourceReqt = (int) Math.ceil((float)
mapResourceReqt/minSlotMemSize)
* minSlotMemSize;
{code}
And this normalization is also 'misusing' the current minimum as the increment
for the normalization.
As a follow up JIRA to this one I plan to fix MR to use the increment instead.
There is value for an AM to know the normalized capacity as based on that it
can decide at allocation request time how to plan its processing distribution
and further allocations (if I ask for 1.2 GB and I'll be getting 2GB I can bump
at planning phase how much will do in that container and correct my subsequent
allocation requests to be less).
Also, are you suggesting to get rid of min/max from the API and make clients
get such info from the config?
> Add multiplier unit to resourcecapabilities
> -------------------------------------------
>
> Key: YARN-689
> URL: https://issues.apache.org/jira/browse/YARN-689
> Project: Hadoop YARN
> Issue Type: Bug
> Components: api, scheduler
> Affects Versions: 2.0.4-alpha
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Attachments: YARN-689.patch, YARN-689.patch, YARN-689.patch,
> YARN-689.patch, YARN-689.patch
>
>
> Currently we overloading the minimum resource value as the actual multiplier
> used by the scheduler.
> Today with a minimum memory set to 1GB, requests for 1.5GB are always
> translated to allocation of 2GB.
> We should decouple the minimum allocation from the multiplier.
> The multiplier should also be exposed to the client via the
> RegisterApplicationMasterResponse
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
