[
https://issues.apache.org/jira/browse/YARN-9067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Yang reopened YARN-9067:
-----------------------------
Reopen for branch-3.2 and branch-3.1 precommit build test.
> YARN Resource Manager is running OOM because of leak of Configuration Object
> ----------------------------------------------------------------------------
>
> Key: YARN-9067
> URL: https://issues.apache.org/jira/browse/YARN-9067
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn-native-services
> Affects Versions: 3.1.0, 3.2.0, 3.1.1
> Reporter: Eric Yang
> Assignee: Eric Yang
> Priority: Major
> Fix For: 3.3.0
>
> Attachments: YARN-9067-branch-3.2.001.patch, YARN-9067.001.patch,
> YARN-9067.002.patch, YARN-9067.003.patch, image-2018-11-27-09-55-16-549.png
>
>
> Resource Manager is running out of memory after every 2-3 days in dev cluster,
> After Analyzing the memory dump , it looks like HDFS is leaking
> configuration object causing YARN RM OOM.
> GC Logs:
> {code:java}
> PSYoungGen total 52736K, used 37813K [0x00000000eab00000,
> 0x00000000eec80000, 0x0000000100000000)
> eden space 39424K, 95% used
> [0x00000000eab00000,0x00000000ecfed620,0x00000000ed180000)
> from space 13312K, 0% used
> [0x00000000edf80000,0x00000000edf80000,0x00000000eec80000)
> to space 13824K, 0% used
> [0x00000000ed180000,0x00000000ed180000,0x00000000edf00000)
> ParOldGen total 699392K, used 699329K [0x00000000c0000000,
> 0x00000000eab00000, 0x00000000eab00000)
> object space 699392K, 99% used
> [0x00000000c0000000,0x00000000eaaf04a8,0x00000000eab00000)
> Metaspace used 98178K, capacity 99932K, committed 100440K, reserved
> 1138688K
> class space used 10481K, capacity 10829K, committed 10880K, reserved
> 1048576K
> {code}
> More than 8K objects of org/apache/Hadoop/Conf and most frequent code path to
> create Hadoop Configuration object is coming from
> org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider and
> all these object are kept in memory, see the attached screenshot for the path
> to GC root for conf object.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
