[
https://issues.apache.org/jira/browse/YARN-2637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14263608#comment-14263608
]
Craig Welch commented on YARN-2637:
-----------------------------------
bq. I think there should at least one AM can be launched in each queue ...
MockRM test config settings
That's been the case since switching to approach 2, some tests need to start >
1 app in a queue ;) In any case, I've removed the MockRM test config settings,
it's only needed in a few tests now, so I'm setting it those tests directly
(done)
bq. -re maximumActiveApplications ... MAXIMUM_ACTIVE_APPLICATIONS_SUFFIX
I removed this new configuration point. It is no longer possible to directly
control how many apps start in a queue since the AM's are not all the same
size, so it's not possible to actually control that now outside of testing (it
was before, not it's not). However, the cases I recall using that were all to
work around the fact that the max am percent wasn't working properly, so
hopefully this won't be missed
(done)
-re null checks in FiCaSchedulerApp constructor
So, the ResourceManager itself checks for null rmapps (ResourceManager.java~
line 830), this is a pre-existing case which is tolerated and I'm not going to
address it. The getAMResourceRequest() can also be null for unmanaged AM's.
I've reduced the null checks for the app to just these two cases but those
checks should remain.
(partly done/remaining should stay as-is)
All the build quality checks and tests are passing, not sure why the overall is
red, think it's a build server issue...
> maximum-am-resource-percent could be violated when resource of AM is >
> minimumAllocation
> ----------------------------------------------------------------------------------------
>
> Key: YARN-2637
> URL: https://issues.apache.org/jira/browse/YARN-2637
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
> Affects Versions: 2.6.0
> Reporter: Wangda Tan
> Assignee: Craig Welch
> Priority: Critical
> Attachments: YARN-2637.0.patch, YARN-2637.1.patch,
> YARN-2637.12.patch, YARN-2637.13.patch, YARN-2637.15.patch,
> YARN-2637.16.patch, YARN-2637.17.patch, YARN-2637.18.patch,
> YARN-2637.19.patch, YARN-2637.2.patch, YARN-2637.20.patch,
> YARN-2637.21.patch, YARN-2637.22.patch, YARN-2637.23.patch,
> YARN-2637.25.patch, YARN-2637.6.patch, YARN-2637.7.patch, YARN-2637.9.patch
>
>
> Currently, number of AM in leaf queue will be calculated in following way:
> {code}
> max_am_resource = queue_max_capacity * maximum_am_resource_percent
> #max_am_number = max_am_resource / minimum_allocation
> #max_am_number_for_each_user = #max_am_number * userlimit * userlimit_factor
> {code}
> And when submit new application to RM, it will check if an app can be
> activated in following way:
> {code}
> for (Iterator<FiCaSchedulerApp> i=pendingApplications.iterator();
> i.hasNext(); ) {
> FiCaSchedulerApp application = i.next();
>
> // Check queue limit
> if (getNumActiveApplications() >= getMaximumActiveApplications()) {
> break;
> }
>
> // Check user limit
> User user = getUser(application.getUser());
> if (user.getActiveApplications() <
> getMaximumActiveApplicationsPerUser()) {
> user.activateApplication();
> activeApplications.add(application);
> i.remove();
> LOG.info("Application " + application.getApplicationId() +
> " from user: " + application.getUser() +
> " activated in queue: " + getQueueName());
> }
> }
> {code}
> An example is,
> If a queue has capacity = 1G, max_am_resource_percent = 0.2, the maximum
> resource that AM can use is 200M, assuming minimum_allocation=1M, #am can be
> launched is 200, and if user uses 5M for each AM (> minimum_allocation). All
> apps can still be activated, and it will occupy all resource of a queue
> instead of only a max_am_resource_percent of a queue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
