Tao Yang created YARN-11766:
-------------------------------
Summary: Implement DELAYED_OR operator in the scheduling process
for Rich Placement Constraint
Key: YARN-11766
URL: https://issues.apache.org/jira/browse/YARN-11766
Project: Hadoop YARN
Issue Type: Sub-task
Components: capacityscheduler
Reporter: Tao Yang
Assignee: Tao Yang
DELAYED_OR operator has been defined in the [Rich Placement Constraints Design
doc|https://issues.apache.org/jira/secure/attachment/12867869/YARN-6592-Rich-Placement-Constraints-Design-V1.pdf],
and is already existing in the API. But haven't been implemented in the
scheduling process yet.
This ticket will implement DELAYED_OR operator in the scheduling process to
support soft placement constraints requirements, according to the description
in design doc:
{code}
DELAYED_OR: Similar to OR, but supports ordering and delays for constraints.
Example : Try for 2 mins to place me to host n1; then try for 3 mins to place
me on rack r1; if everything else fails, place me on any node. The following
delay criteria can be specified to control the following:
* Number of units of delay for each constraint in the expression.
* Units of delay can be either missed opportunities or wall clock time.
* Ability to reset units of delay after one of the allocations gets allocated
(similar to today’s resetRackDelay).
{code}
The last delay criteria has been improved to make it usable: It's reasonable to
reset units of delay after one of the allocations gets allocated for
OPPORTUNITIES delay-unit, but for MILLISECONDS delay-unit, the update time
should be reset only when the request is updated, otherwise the request with
large size will be satisfied slowly: one by one satisfied with the specified
delay interval. For example, if we specified the DELAY_OR constraint for a
request with 100 containers and the delay interval is 1 minute, it is expected
to be satisfied after 1 minute instead of 100 minutes for all the 100
containers.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
