[
https://issues.apache.org/jira/browse/YARN-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135340#comment-16135340
]
ASF GitHub Bot commented on YARN-2162:
--------------------------------------
Github user kambatla commented on a diff in the pull request:
https://github.com/apache/hadoop/pull/261#discussion_r134268440
--- Diff:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/FairScheduler.md
---
@@ -85,9 +85,9 @@ The allocation file must be in XML format. The format
contains five types of ele
* **minResources**: minimum resources the queue is entitled to, in the
form "X mb, Y vcores". For the single-resource fairness policy, the vcores
value is ignored. If a queue's minimum share is not satisfied, it will be
offered available resources before any other queue under the same parent. Under
the single-resource fairness policy, a queue is considered unsatisfied if its
memory usage is below its minimum memory share. Under dominant resource
fairness, a queue is considered unsatisfied if its usage for its dominant
resource with respect to the cluster capacity is below its minimum share for
that resource. If multiple queues are unsatisfied in this situation, resources
go to the queue with the smallest ratio between relevant resource usage and
minimum. Note that it is possible that a queue that is below its minimum may
not immediately get up to its minimum when it submits an application, because
already-running jobs may be using those resources.
- * **maxResources**: maximum resources a queue is allowed, in the form
"X mb, Y vcores". A queue will never be assigned a container that would put its
aggregate usage over this limit.
+ * **maxResources**: maximum resources a queue is allowed, in the form
"X mb, Y vcores". A queue will never be assigned a container that would put its
aggregate usage over this limit. Set it to a percentage of all cluster
resources in form of "50%" or "50% memory, 60% vcore".
- * **maxChildResources**: maximum resources an ad hoc child queue is
allowed, in the form "X mb, Y vcores". Any ad hoc queue that is a direct child
of a queue with this property set will have it's maxResources property set
accordingly.
+ * **maxChildResources**: maximum resources an ad hoc child queue is
allowed, in the form "X mb, Y vcores". Any ad hoc queue that is a direct child
of a queue with this property set will have it's maxResources property set
accordingly. Set it to a percentage of all cluster resources in form of "50%"
or "50% memory, 60% vcore".
--- End diff --
Similar to the above config?
> add ability in Fair Scheduler to optionally configure maxResources in terms
> of percentage
> -----------------------------------------------------------------------------------------
>
> Key: YARN-2162
> URL: https://issues.apache.org/jira/browse/YARN-2162
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: fairscheduler, scheduler
> Reporter: Ashwin Shankar
> Assignee: Yufei Gu
> Labels: scheduler
> Attachments: YARN-2162.001.patch, YARN-2162.002.patch
>
>
> minResources and maxResources in fair scheduler configs are expressed in
> terms of absolute numbers X mb, Y vcores.
> As a result, when we expand or shrink our hadoop cluster, we need to
> recalculate and change minResources/maxResources accordingly, which is pretty
> inconvenient.
> We can circumvent this problem if we can optionally configure these
> properties in terms of percentage of cluster capacity.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
