[ https://issues.apache.org/jira/browse/YARN-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16115205#comment-16115205 ]
YunFan Zhou edited comment on YARN-6361 at 8/5/17 1:49 AM: ----------------------------------------------------------- [~yufeigu] Thank Yufei. Sorry, I'm off the subject. But either way, the efficiency of raising the *fetchAppsWithDemand* is something that must be done. I have thought about the optimal method for two days and tested my thoughts today. Thank you very much for your confidence in me. was (Author: daemon): [~yufeigu] Thank Yufei. Sorry, I'm off the subject. But either way, the efficiency of raising the *fetchAppsWithDemand *is something that must be done. I have thought about the optimal method for two days and tested my thoughts today. Thank you very much for your confidence in me. > FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big > queues > -------------------------------------------------------------------------------- > > Key: YARN-6361 > URL: https://issues.apache.org/jira/browse/YARN-6361 > Project: Hadoop YARN > Issue Type: Sub-task > Reporter: Miklos Szegedi > Assignee: YunFan Zhou > Priority: Minor > Attachments: dispatcherthread.png, threads.png > > > FSLeafQueue.fetchAppsWithDemand sorts the applications by the current policy. > Most of the time is spent in FairShareComparator.compare. We could improve > this by doing the calculations outside the sort loop {{(O\(n\))}} and we > sorted by a fixed number inside instead {{O(n*log\(n\))}}. This could be an > performance issue when there are huge number of applications in a single > queue. The attachments shows the performance impact when there are 10k > applications in one queue. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org