[
https://issues.apache.org/jira/browse/YARN-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16643713#comment-16643713
]
Íñigo Goiri commented on YARN-8827:
-----------------------------------
Some minor comments:
* NodeStatusPBImpl extra line before addAppUtilizations.
* NodeStatusPBImpl#63 fits in one line.
* In NodeStatusPBImpl#128, I would extract the key and the value for
readability (e.g., appId = entry.getKey()).
* Similar for initAppUtilizations.
* Can we use logger in ResourceUtilizationAggregator and use the {} pattern for
the logs?
* We do a lot the pattern of new empty utilization, I think we could create
ResourceUtilization.newInstance() setting all to 0, or newZero(). Actually the
pattern for utilization maps is always the same. At some point it might be
useful to have a UtilizationMap.
* In MockNM can we keep the old method passing null by default for app
utilization and just add the new method instead of replacing?
* We can resolve a few of the remaining checkstyle, I think it is fine to make
the fields private for example.
Other than this minor things, this LGTM.
Whenever we open the follow-up JIRA that uses this, can you link it to this one?
> Plumb per app, per user and per queue resource utilization from the NM to RM
> ----------------------------------------------------------------------------
>
> Key: YARN-8827
> URL: https://issues.apache.org/jira/browse/YARN-8827
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Arun Suresh
> Assignee: Arun Suresh
> Priority: Major
> Attachments: YARN-8827-YARN-1011.01.patch,
> YARN-8827-YARN-1011.02.patch, YARN-8827-YARN-1011.03.patch,
> YARN-8827-YARN-1011.04.patch, YARN-8827-YARN-1011.05.patch
>
>
> Opportunistic Containers for OverAllocation need to be allocated to pending
> applications in some fair manner. Rather than evaluating queue and user
> resource usage (allocated resource usage) and comparing against queue and
> user limits to decide the allocation, it might make more sense to use a
> snapshot of actual resource utilization of the queue and user.
> To facilitate this, this JIRA proposes to aggregate per user, per app (and
> maybe per queue) resource utilization in addition to aggregated Container and
> Node Utilization and send it along with the NM heartbeat. It should be fairly
> inexpensive to aggregate - since it can be performed in the same loop of the
> {{ContainersMonitorImpl}}'s Monitoring thread.
> A snapshot aggregate can be made every couple of seconds in the RM. This
> instantaneous resource utilization should be used to decide if Opportunistic
> containers can be allocated to an App, Queue or User.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
