This is an automated email from the ASF dual-hosted git repository. snemeth pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/hadoop.git
The following commit(s) were added to refs/heads/trunk by this push: new 5446308 YARN-9326. Fair Scheduler configuration defaults are not documented in case of min and maxResources. Contributed by Adam Antal 5446308 is described below commit 5446308360f57cb98c54c416231788ba9ae332f8 Author: Szilard Nemeth <snem...@apache.org> AuthorDate: Mon Jul 15 13:28:01 2019 +0200 YARN-9326. Fair Scheduler configuration defaults are not documented in case of min and maxResources. Contributed by Adam Antal --- .../hadoop-yarn-site/src/site/markdown/FairScheduler.md | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/FairScheduler.md b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/FairScheduler.md index 5f9e779..991796a 100644 --- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/FairScheduler.md +++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/FairScheduler.md @@ -86,13 +86,19 @@ The allocation file must be in XML format. The format contains five types of ele * **Queue elements**: which represent queues. Queue elements can take an optional attribute 'type', which when set to 'parent' makes it a parent queue. This is useful when we want to create a parent queue without configuring any leaf queues. Each queue element may contain the following properties: - * **minResources**: minimum resources the queue is entitled to, in the form of "X mb, Y vcores" or "vcores=X, memory-mb=Y". The latter form is required when specifying resources other than memory and CPU. For the single-resource fairness policy, the vcores value is ignored. If a queue's minimum share is not satisfied, it will be offered available resources before any other queue under the same parent. Under the single-resource fairness policy, a queue is considered unsatisfied if its [...] + * **minResources**: minimum resources the queue is entitled to. For the single-resource fairness policy, only the memory is used, other resources are ignored. If a queue's minimum share is not satisfied, it will be offered available resources before any other queue under the same parent. Under the single-resource fairness policy, a queue is considered unsatisfied if its memory usage is below its minimum memory share. Under dominant resource fairness, a queue is considered unsatisfied [...] - * **maxResources**: maximum resources a queue will allocated, expressed in the form of "X%", "X% cpu, Y% memory", "X mb, Y vcores", or "vcores=X, memory-mb=Y". The last form is required when specifying resources other than memory and CPU. In the last form, X and Y can either be a percentage or an integer resource value without units. In the latter case the units will be inferred from the default units configured for that resource. A queue will not be assigned a container that would p [...] + * **maxResources**: maximum resources a queue can be allocated. A queue will not be assigned a container that would put its aggregate usage over this limit. This limit is enforced recursively, the queue will not be assigned a container if that assignment would put the queue or its parent(s) over the maximum resources. - * **maxContainerAllocation**: maximum resources a queue can allocate for a single container, expressed in the form of "X mb, Y vcores" or "vcores=X, memory-mb=Y". The latter form is required when specifying resources other than memory and CPU. If the property is not set it's value is inherited from a parent queue. It's default value is **yarn.scheduler.maximum-allocation-mb**. Cannot be higher than **maxResources**. This property is invalid for root queue. + * **maxContainerAllocation**: maximum resources a queue can allocate for a single container. If the property is not set it's value is inherited from a parent queue. The default values are **yarn.scheduler.maximum-allocation-mb** and **yarn.scheduler.maximum-allocation-vcores**. Cannot be higher than **maxResources**. This property is invalid for root queue. - * **maxChildResources**: maximum resources an ad hoc child queue will allocated, expressed in the form of "X%", "X% cpu, Y% memory", "X mb, Y vcores", or "vcores=X, memory-mb=Y". The last form is required when specifying resources other than memory and CPU. In the last form, X and Y can either be a percentage or an integer resource value without units. In the latter case the units will be inferred from the default units configured for that resource. An ad hoc child queue will not be [...] + * **maxChildResources**: maximum resources an ad hoc child queue can be allocated. A child queue limit is enforced recursively and thus will not be assigned a container if that assignment would put the child queue or its parent(s) over the maximum resources. + + * For minResources, maxResources, maxContainerAllocation and maxChildResources properties one can give the parameter in one of the following formats: + + * *Old format*: "X mb, Y vcores", "X% cpu, Y% memory", "X%". When not a single percentage is provided, configuring both memory and cpu is mandatory, while other resource types are ignored, and set to zero. + + * *New format (recommended)*: "vcores=X, memory-mb=Y" or "vcores=X%, memory-mb=Y%". As seen, in this format one can give either a percentage or an integer resource value without units. In the latter case the units will be inferred from the default units configured for that resource. This format is required when resources other than memory and CPU are specified. Any unspecified resource will be set to 0 in case of minResources, or to the maximum for that resource in case of ma [...] * **maxRunningApps**: limit the number of apps from the queue to run at once --------------------------------------------------------------------- To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-commits-h...@hadoop.apache.org