[
https://issues.apache.org/jira/browse/YARN-3215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074330#comment-15074330
]
Wangda Tan commented on YARN-3215:
----------------------------------
Hi [~Naganarasimha],
bq. Any concerns in doing this way
The biggest concern is it's an API change: downstream projects have to modify
their code, and we don't know if our proposed API is what they want. I would
suggest to delay the API change to 2.9 according to short timeframe for 2.8.
bq. First the app needs to specify the resource requests specifying which
partitions they want but IIUC first they send a empty request to get the head
room and then start scheduling internally what tasks (diff priority) to run ...
I'm not sure if it is correct for our several biggest applications, for
example: MR/Spark/Tez/Slider. The first resource request of an app should be
its AM request, we can at least get *headroom of the AM's partition*. And
returning headroom of what AM requests seems nature to me. Just like purchasing
a car: you won't know the quote until you decided target model(s).
Thoughts?
> Respect labels in CapacityScheduler when computing headroom
> -----------------------------------------------------------
>
> Key: YARN-3215
> URL: https://issues.apache.org/jira/browse/YARN-3215
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler
> Reporter: Wangda Tan
> Assignee: Naganarasimha G R
>
> In existing CapacityScheduler, when computing headroom of an application, it
> will only consider "non-labeled" nodes of this application.
> But it is possible the application is asking for labeled resources, so
> headroom-by-label (like 5G resource available under node-label=red) is
> required to get better resource allocation and avoid deadlocks such as
> MAPREDUCE-5928.
> This JIRA could involve both API changes (such as adding a
> label-to-available-resource map in AllocateResponse) and also internal
> changes in CapacityScheduler.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
