[ https://issues.apache.org/jira/browse/YUNIKORN-807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402739#comment-17402739 ]
Sunil G commented on YUNIKORN-807: ---------------------------------- Thanks [~ccondit] Overall, I think this approach looks good and will definitely optimize our node scorer algorithm. Awesome performance numbers as well. Let's target for the next release. A couple of high levels thoughts: 1. DRF implementation is poor in YuniKorn today. This will work as a good base. And I think we can improve that in a follow up jira where we need to find dominant or highly utilized resources in a node in a better way 2. Listeners model/approach looks good. Do you see a possible hotspot in the nodeUpdated event handling? Thanks for the PR and I will review more in PR. > Improve performance of node sorting > ----------------------------------- > > Key: YUNIKORN-807 > URL: https://issues.apache.org/jira/browse/YUNIKORN-807 > Project: Apache YuniKorn > Issue Type: Improvement > Components: core - scheduler > Reporter: Craig Condit > Assignee: Craig Condit > Priority: Major > Labels: pull-request-available > Attachments: Node Sorting Performance Improvement.pdf > > > YuniKorn currently sorts all nodes on demand whenever scheduling of a > container occurs. This causes significant performance degradation as the > number of nodes increases. > If we replace the on-demand sorting with a B-Tree sorted proactively, we can > improve performance considerably. > This is a similar approach to YUNIKORN-21, but without the associated > behavioral changes. > I've attached a design document with the details of the approach and the > performance improvement gained. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org For additional commands, e-mail: issues-h...@yunikorn.apache.org