[
https://issues.apache.org/jira/browse/YARN-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14542895#comment-14542895
]
Rohit Agarwal commented on YARN-3633:
-------------------------------------
Can you elaborate the scenario where the AMs would come up but the containers
would not? There is still a cluster-wide maxAMShare which is, let's say, 0.5
times the cluster-capacity.
A dry run of my proposed change:
Cluster Resource: 20GB
Applications submitted to 4 queues simultaneously - queue a, b, c, and d. Each
application requests AMs of size 3GB and containers of size 3GB.
Fair share for each queue = 5GB.
maxAMShare is 0.5
Let's say {{SomeMinimumSizeEnoughToRunOneContainer}} is 3GB.
queue a will start AM1. (maxAMShare = max(0.5*5GB, 3GB) = 3GB). cluster-wide
AMShare = 3GB < 0.5*20GB
queue b will start AM2. (maxAMShare = max(0.5*5GB, 3GB) = 3GB). cluster-wide
AMShare = 6GB < 0.5*20GB
queue c will start AM3. (maxAMShare = max(0.5*5GB, 3GB) = 3GB). cluster-wide
AMShare = 9GB < 0.5*20GB
FS will try to run AM4 on queue d. But now it would hit the cluster-wide
maxAMShare limit. So, nothing will run there. Then, FS will try to run
something on queue a (or b or c) - and so the application1 container would
start.
This would repeat. FS will try to run AM4 on queue d. It would again hit the
cluster-wide maxAMShare limit. It would then try to run something on queue b
(or c) - and so application2 container would start.
And so on.
Finally one of app1, app2, app3 AM would finish. At which time FS should
schedule AM4 on queue d.
I agree that queue d is not getting its fair share till one of app1, app2 and
app3 complete (and that is why I am unsure how my proposed change would work
when preemption is enabled.) But I think it is better than not scheduling
anything?
> With Fair Scheduler, cluster can logjam when there are too many queues
> ----------------------------------------------------------------------
>
> Key: YARN-3633
> URL: https://issues.apache.org/jira/browse/YARN-3633
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
> Affects Versions: 2.6.0
> Reporter: Rohit Agarwal
> Assignee: Rohit Agarwal
> Priority: Critical
>
> It's possible to logjam a cluster by submitting many applications at once in
> different queues.
> For example, let's say there is a cluster with 20GB of total memory. Let's
> say 4 users submit applications at the same time. The fair share of each
> queue is 5GB. Let's say that maxAMShare is 0.5. So, each queue has at most
> 2.5GB memory for AMs. If all the users requested AMs of size 3GB - the
> cluster logjams. Nothing gets scheduled even when 20GB of resources are
> available.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
