[ https://issues.apache.org/jira/browse/MAPREDUCE-5351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13819132#comment-13819132 ]
Chris Nauroth commented on MAPREDUCE-5351: ------------------------------------------ Hi, [~jandyfish]. Thank you for reporting this and posting a patch. Have you seen the related issue MAPREDUCE-5508? I believe that fixes the bug you saw. The MAPREDUCE-5508 patch has already been reviewed and committed. We tested it at scale (hundreds of nodes/thousands of job executions) and saw no occurrences of the leak. > JobTracker memory leak caused by CleanupQueue reopening FileSystem > ------------------------------------------------------------------ > > Key: MAPREDUCE-5351 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5351 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker > Affects Versions: 1.1.2 > Reporter: Sandy Ryza > Assignee: Sandy Ryza > Priority: Critical > Fix For: 1-win, 1.2.1 > > Attachments: JobInProgress_JobHistory.patch, MAPREDUCE-5351-1.patch, > MAPREDUCE-5351-2.patch, MAPREDUCE-5351-addendum-1.patch, > MAPREDUCE-5351-addendum.patch, MAPREDUCE-5351.patch > > > When a job is completed, closeAllForUGI is called to close all the cached > FileSystems in the FileSystem cache. However, the CleanupQueue may run after > this occurs and call FileSystem.get() to delete the staging directory, adding > a FileSystem to the cache that will never be closed. > People on the user-list have reported this causing their JobTrackers to OOME > every two weeks. -- This message was sent by Atlassian JIRA (v6.1#6144)