[
https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13402244#comment-13402244
]
Devaraj K commented on MAPREDUCE-4379:
--------------------------------------
{code:title=ContainerLocalizer.java|borderStyle=solid}
this.appDirs =
new LocalDirAllocator(String.format(APPCACHE_CTXT_FMT, appId));
this.userDirs =
new LocalDirAllocator(String.format(USERCACHE_CTXT_FMT, appId));
this.pendingResources = new HashMap<LocalResource,Future<Path>>();
{code}
Here for every application during localization, it creates two
LocalDirAllocator instances.
{code:title=LocalDirAllocator.java|borderStyle=solid}
private AllocatorPerContext obtainContext(String contextCfgItemName) {
synchronized (contexts) {
AllocatorPerContext l = contexts.get(contextCfgItemName);
if (l == null) {
contexts.put(contextCfgItemName,
(l = new AllocatorPerContext(contextCfgItemName)));
}
return l;
}
}
{code}
Those two instances will internally creates AllocatorPerContext instances and
add those into contexts while obtaining contexts. It will keep on adding for
every application and no where else these are getting removed from the map. It
is leading to OOM after running for some time.
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to
> org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 2.0.0-alpha, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Critical
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap
> space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at
> org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at
> org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for
> container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java
> heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at
> com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at
> com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at
> com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at
> com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at
> com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at
> com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at
> org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
