[
https://issues.apache.org/jira/browse/YARN-3482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14500097#comment-14500097
]
Karthik Kambatla commented on YARN-3482:
----------------------------------------
bq. With this and the containers utilization, we can estimate the utilization
of external processes.
True, but I fear that will be too conservative. If we go that route, HBase
RegionServers could grow aggressively and adversely affect resources under
Yarn. By having an interface for available resources, we ensure Yarn
aggressively schedules work to claim all available resources. Changing these
available resources could be through a secure interface admins or a white-list
of processes can access.
> Report NM available resources in heartbeat
> ------------------------------------------
>
> Key: YARN-3482
> URL: https://issues.apache.org/jira/browse/YARN-3482
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager, resourcemanager
> Affects Versions: 2.7.0
> Reporter: Inigo Goiri
> Original Estimate: 504h
> Remaining Estimate: 504h
>
> NMs are usually collocated with other processes like HDFS, Impala or HBase.
> To manage this scenario correctly, YARN should be aware of the actual
> available resources. The proposal is to have an interface to dynamically
> change the available resources and report this to the RM in every heartbeat.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
