[ https://issues.apache.org/jira/browse/MAPREDUCE-4715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Ryakhovskiy updated MAPREDUCE-4715: ----------------------------------------- Attachment: VisualVM_screenshot.png VisualVM screenshot attached > Heap memory growing when JVM reuse option enabled > ------------------------------------------------- > > Key: MAPREDUCE-4715 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4715 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: job submission > Affects Versions: 0.20.2 > Environment: Linux, cdh3u5 > Reporter: Boris Ryakhovskiy > Priority: Minor > Attachments: VisualVM_screenshot.png > > > When mapred.job.reuse.jvm.num.tasks option is set to 100 or more tasks, JVM > fails with "java.lang.OutOfMemoryError: Java heap space" error message. Jobs > itself are small and job heap size is set to -Xmx200m . > It looks like the reason of the issue is a FileSystem$Cache object which > collects JobConf objects from all jobs which were executed in current JVM. GC > root for the FileSystem$Cache is ApplicationShutdownHooks class (cannot > attach screenshot from visualvm) > In my case each of JobConf objects allocates ~500K (50M total in case of 100 > jobs). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira