[ https://issues.apache.org/jira/browse/YARN-6679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16043321#comment-16043321 ]
Wangda Tan commented on YARN-6679: ---------------------------------- [~daryn]/[~templedf], I just tried to test the patch using SLS: (a generated trace which consists of 8K nodes, each node has 128GB memory, 400 concurrent running apps, size of each app varies). >From my test, YARN scheduler can allocate 2700-3100 containers / sec on latest >trunk with and without the patch, I didn't see obvious regression however I >didn't see obvious improvement either. Not sure if it is the same for >branch-2.8 as we have already fixed some performance issues on trunk. I don't have chance to look into the issue since theoretically this patch should improve performance. Not sure if we hit any other bottleneck, [~daryn], could you share size / allocation-throughput of your cluster? > Reduce Resource instance overhead via non-PBImpl > ------------------------------------------------ > > Key: YARN-6679 > URL: https://issues.apache.org/jira/browse/YARN-6679 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager > Affects Versions: 2.8.0 > Reporter: Daryn Sharp > Assignee: Daryn Sharp > Fix For: 2.9.0, 3.0.0-alpha4 > > Attachments: YARN-6679.2.branch-2.patch, YARN-6679.2.trunk.patch, > YARN-6679.3.branch-2.patch, YARN-6679.3.trunk.patch, > YARN-6679.branch-2.patch, YARN-6679.trunk.patch > > > Creating and using transient PB-based Resource instances during scheduling is > very expensive. The overhead can be transparently reduced by internally > using lightweight non-PB based instances. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org