Peter Bacsko created YUNIKORN-1715:
--------------------------------------
Summary: Yunikorn performance improvements
Key: YUNIKORN-1715
URL: https://issues.apache.org/jira/browse/YUNIKORN-1715
Project: Apache YuniKorn
Issue Type: Improvement
Components: core - scheduler
Reporter: Peter Bacsko
There are some methods/functions in Yunikorn which are called frequently and
often unnecessarily. On a large, busy cluster, eliminating these calls can
result in faster scheduling cycle, therefore better throughput.
In the listed cases below, we can re-use a previously computed value and the
expensive copy/sort phase can be eliminated completely.
*Retrieving node iterators*: in
{{baseNodeCollection.getNodeIteratorInternal()}}, we always clone the tree of
sorted nodes, then we build a slice. The node tree is only modified when a node
gets a new score.
*Queue sorting*: only need sorting in the following events:
* Allocated resource changed in one of the child queues (most common)
* Pending resource changed from 0 to "n", or from "n" to 0 (affects filtering)
* Child queue got stopped (affects filtering)
* Child queue structure changed on config update
*Application sorting*: in {{Queue.TryAllocate()}} and
{{Queue.TryPlaceholderAllocate()}}, {{sortApplications()}} always runs. In
every iteration, it calls {{Queue.GetCopyOfApps()}}. It only has to run if
something relevant happens from the sorting POV:
* Application added/removed
* Ask added to an application
* Ask max priority changed in at least one application
* Allocated resource changed in at least one application
*Request sorting*: request (ask) sorting is only necessary when the following
occurs:
* Ask added
* pendingAskRepeat gets 0 in an ask
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
