[
https://issues.apache.org/jira/browse/YARN-291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619298#comment-13619298
]
Alejandro Abdelnur commented on YARN-291:
-----------------------------------------
bq. How so? In fact ...
Changes in NM capacity triggered from outside of the regular scheduling would
unbalance existing distribution of allocations potentially triggering
preemption. You'd need to handle this specially in the RM/scheduler to handle
such scenarios.
bq. First this requires modifying existing applications to support new request
API...
Not really, the API would be augmented in a compatible way to support what I'm
describing, existing applications should not be affected as they don't use this
new functionality.
bq. Second, RM scheduling is fundamentally high latency ....
It depends how you design you AM that handles unmanaged containers. You could
request several small resources on peak and then release them as you don't need
them.
bq. Note, the existing simple patch doesn't change existing API/protocols.
It is adding a new one, that is a change.
> Dynamic resource configuration
> ------------------------------
>
> Key: YARN-291
> URL: https://issues.apache.org/jira/browse/YARN-291
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: nodemanager, scheduler
> Reporter: Junping Du
> Assignee: Junping Du
> Labels: features
> Attachments: Elastic Resources for YARN-v0.2.pdf,
> YARN-291-AddClientRMProtocolToSetNodeResource-03.patch,
> YARN-291-all-v1.patch, YARN-291-core-HeartBeatAndScheduler-01.patch,
> YARN-291-JMXInterfaceOnNM-02.patch,
> YARN-291-OnlyUpdateWhenResourceChange-01-fix.patch,
> YARN-291-YARNClientCommandline-04.patch
>
>
> The current Hadoop YARN resource management logic assumes per node resource
> is static during the lifetime of the NM process. Allowing run-time
> configuration on per node resource will give us finer granularity of resource
> elasticity. This allows Hadoop workloads to coexist with other workloads on
> the same hardware efficiently, whether or not the environment is virtualized.
> About more background and design details, please refer: HADOOP-9165.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
