[jira] [Updated] (MAPREDUCE-5351) JobTracker memory leak caused by CleanupQueue reopening FileSystem
[ https://issues.apache.org/jira/browse/MAPREDUCE-5351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sanghyun Yun updated MAPREDUCE-5351: Assignee: Sandy Ryza (was: Sanghyun Yun) > JobTracker memory leak caused by CleanupQueue reopening FileSystem > -- > > Key: MAPREDUCE-5351 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5351 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 1.1.2 >Reporter: Sandy Ryza >Assignee: Sandy Ryza >Priority: Critical > Fix For: 1-win, 1.2.1 > > Attachments: JobInProgress_JobHistory.patch, MAPREDUCE-5351-1.patch, > MAPREDUCE-5351-2.patch, MAPREDUCE-5351-addendum-1.patch, > MAPREDUCE-5351-addendum.patch, MAPREDUCE-5351.patch > > > When a job is completed, closeAllForUGI is called to close all the cached > FileSystems in the FileSystem cache. However, the CleanupQueue may run after > this occurs and call FileSystem.get() to delete the staging directory, adding > a FileSystem to the cache that will never be closed. > People on the user-list have reported this causing their JobTrackers to OOME > every two weeks. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (MAPREDUCE-5351) JobTracker memory leak caused by CleanupQueue reopening FileSystem
[ https://issues.apache.org/jira/browse/MAPREDUCE-5351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jandyfish updated MAPREDUCE-5351: - Attachment: JobInProgress_JobHistory.patch find a bug : open a filesystem when JobHistory create userLogFile,but this filesystem instance will not be close,and it still cache in FileSystem.cache。 i comminited this svn patch JobInProcess_JobHistory.path。i hope it can help. > JobTracker memory leak caused by CleanupQueue reopening FileSystem > -- > > Key: MAPREDUCE-5351 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5351 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 1.1.2 >Reporter: Sandy Ryza >Assignee: Sandy Ryza >Priority: Critical > Fix For: 1-win, 1.2.1 > > Attachments: JobInProgress_JobHistory.patch, MAPREDUCE-5351-1.patch, > MAPREDUCE-5351-2.patch, MAPREDUCE-5351-addendum-1.patch, > MAPREDUCE-5351-addendum.patch, MAPREDUCE-5351.patch > > > When a job is completed, closeAllForUGI is called to close all the cached > FileSystems in the FileSystem cache. However, the CleanupQueue may run after > this occurs and call FileSystem.get() to delete the staging directory, adding > a FileSystem to the cache that will never be closed. > People on the user-list have reported this causing their JobTrackers to OOME > every two weeks. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-5351) JobTracker memory leak caused by CleanupQueue reopening FileSystem
[ https://issues.apache.org/jira/browse/MAPREDUCE-5351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated MAPREDUCE-5351: - Fix Version/s: 1-win I have also committed this patch branch-1-win. > JobTracker memory leak caused by CleanupQueue reopening FileSystem > -- > > Key: MAPREDUCE-5351 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5351 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 1.1.2 >Reporter: Sandy Ryza >Assignee: Sandy Ryza >Priority: Critical > Fix For: 1-win, 1.2.1 > > Attachments: MAPREDUCE-5351-1.patch, MAPREDUCE-5351-2.patch, > MAPREDUCE-5351-addendum-1.patch, MAPREDUCE-5351-addendum.patch, > MAPREDUCE-5351.patch > > > When a job is completed, closeAllForUGI is called to close all the cached > FileSystems in the FileSystem cache. However, the CleanupQueue may run after > this occurs and call FileSystem.get() to delete the staging directory, adding > a FileSystem to the cache that will never be closed. > People on the user-list have reported this causing their JobTrackers to OOME > every two weeks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5351) JobTracker memory leak caused by CleanupQueue reopening FileSystem
[ https://issues.apache.org/jira/browse/MAPREDUCE-5351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley updated MAPREDUCE-5351: -- Resolution: Fixed Status: Resolved (was: Patch Available) Both patch and addendum patch have been applied by Arun, to both branch-1.2 and branch-1. Resolving as Fixed. Thanks, Sandy! > JobTracker memory leak caused by CleanupQueue reopening FileSystem > -- > > Key: MAPREDUCE-5351 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5351 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 1.1.2 >Reporter: Sandy Ryza >Assignee: Sandy Ryza >Priority: Critical > Fix For: 1.2.1 > > Attachments: MAPREDUCE-5351-1.patch, MAPREDUCE-5351-2.patch, > MAPREDUCE-5351-addendum-1.patch, MAPREDUCE-5351-addendum.patch, > MAPREDUCE-5351.patch > > > When a job is completed, closeAllForUGI is called to close all the cached > FileSystems in the FileSystem cache. However, the CleanupQueue may run after > this occurs and call FileSystem.get() to delete the staging directory, adding > a FileSystem to the cache that will never be closed. > People on the user-list have reported this causing their JobTrackers to OOME > every two weeks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5351) JobTracker memory leak caused by CleanupQueue reopening FileSystem
[ https://issues.apache.org/jira/browse/MAPREDUCE-5351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated MAPREDUCE-5351: -- Attachment: MAPREDUCE-5351-addendum-1.patch > JobTracker memory leak caused by CleanupQueue reopening FileSystem > -- > > Key: MAPREDUCE-5351 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5351 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 1.1.2 >Reporter: Sandy Ryza >Assignee: Sandy Ryza >Priority: Critical > Fix For: 1.2.1 > > Attachments: MAPREDUCE-5351-1.patch, MAPREDUCE-5351-2.patch, > MAPREDUCE-5351-addendum-1.patch, MAPREDUCE-5351-addendum.patch, > MAPREDUCE-5351.patch > > > When a job is completed, closeAllForUGI is called to close all the cached > FileSystems in the FileSystem cache. However, the CleanupQueue may run after > this occurs and call FileSystem.get() to delete the staging directory, adding > a FileSystem to the cache that will never be closed. > People on the user-list have reported this causing their JobTrackers to OOME > every two weeks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5351) JobTracker memory leak caused by CleanupQueue reopening FileSystem
[ https://issues.apache.org/jira/browse/MAPREDUCE-5351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated MAPREDUCE-5351: -- Status: Patch Available (was: Reopened) > JobTracker memory leak caused by CleanupQueue reopening FileSystem > -- > > Key: MAPREDUCE-5351 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5351 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 1.1.2 >Reporter: Sandy Ryza >Assignee: Sandy Ryza >Priority: Critical > Fix For: 1.2.1 > > Attachments: MAPREDUCE-5351-1.patch, MAPREDUCE-5351-2.patch, > MAPREDUCE-5351-addendum.patch, MAPREDUCE-5351.patch > > > When a job is completed, closeAllForUGI is called to close all the cached > FileSystems in the FileSystem cache. However, the CleanupQueue may run after > this occurs and call FileSystem.get() to delete the staging directory, adding > a FileSystem to the cache that will never be closed. > People on the user-list have reported this causing their JobTrackers to OOME > every two weeks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5351) JobTracker memory leak caused by CleanupQueue reopening FileSystem
[ https://issues.apache.org/jira/browse/MAPREDUCE-5351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated MAPREDUCE-5351: -- Attachment: MAPREDUCE-5351-addendum.patch > JobTracker memory leak caused by CleanupQueue reopening FileSystem > -- > > Key: MAPREDUCE-5351 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5351 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 1.1.2 >Reporter: Sandy Ryza >Assignee: Sandy Ryza >Priority: Critical > Fix For: 1.2.1 > > Attachments: MAPREDUCE-5351-1.patch, MAPREDUCE-5351-2.patch, > MAPREDUCE-5351-addendum.patch, MAPREDUCE-5351.patch > > > When a job is completed, closeAllForUGI is called to close all the cached > FileSystems in the FileSystem cache. However, the CleanupQueue may run after > this occurs and call FileSystem.get() to delete the staging directory, adding > a FileSystem to the cache that will never be closed. > People on the user-list have reported this causing their JobTrackers to OOME > every two weeks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5351) JobTracker memory leak caused by CleanupQueue reopening FileSystem
[ https://issues.apache.org/jira/browse/MAPREDUCE-5351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-5351: - Resolution: Fixed Fix Version/s: 1.2.1 Status: Resolved (was: Patch Available) I just committed this. Thanks [~sandyr]! [~tucu00] - Sorry, I didn't wait longer to unblock Matt for the release. Please file a follow-on if you have any concernse. Thanks, and sorry again! > JobTracker memory leak caused by CleanupQueue reopening FileSystem > -- > > Key: MAPREDUCE-5351 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5351 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 1.1.2 >Reporter: Sandy Ryza >Assignee: Sandy Ryza >Priority: Critical > Fix For: 1.2.1 > > Attachments: MAPREDUCE-5351-1.patch, MAPREDUCE-5351-2.patch, > MAPREDUCE-5351.patch > > > When a job is completed, closeAllForUGI is called to close all the cached > FileSystems in the FileSystem cache. However, the CleanupQueue may run after > this occurs and call FileSystem.get() to delete the staging directory, adding > a FileSystem to the cache that will never be closed. > People on the user-list have reported this causing their JobTrackers to OOME > every two weeks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5351) JobTracker memory leak caused by CleanupQueue reopening FileSystem
[ https://issues.apache.org/jira/browse/MAPREDUCE-5351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated MAPREDUCE-5351: -- Attachment: MAPREDUCE-5351-2.patch > JobTracker memory leak caused by CleanupQueue reopening FileSystem > -- > > Key: MAPREDUCE-5351 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5351 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 1.1.2 >Reporter: Sandy Ryza >Assignee: Sandy Ryza >Priority: Critical > Attachments: MAPREDUCE-5351-1.patch, MAPREDUCE-5351-2.patch, > MAPREDUCE-5351.patch > > > When a job is completed, closeAllForUGI is called to close all the cached > FileSystems in the FileSystem cache. However, the CleanupQueue may run after > this occurs and call FileSystem.get() to delete the staging directory, adding > a FileSystem to the cache that will never be closed. > People on the user-list have reported this causing their JobTrackers to OOME > every two weeks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5351) JobTracker memory leak caused by CleanupQueue reopening FileSystem
[ https://issues.apache.org/jira/browse/MAPREDUCE-5351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated MAPREDUCE-5351: -- Status: Patch Available (was: Open) > JobTracker memory leak caused by CleanupQueue reopening FileSystem > -- > > Key: MAPREDUCE-5351 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5351 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 1.1.2 >Reporter: Sandy Ryza >Assignee: Sandy Ryza >Priority: Critical > Attachments: MAPREDUCE-5351-1.patch, MAPREDUCE-5351.patch > > > When a job is completed, closeAllForUGI is called to close all the cached > FileSystems in the FileSystem cache. However, the CleanupQueue may run after > this occurs and call FileSystem.get() to delete the staging directory, adding > a FileSystem to the cache that will never be closed. > People on the user-list have reported this causing their JobTrackers to OOME > every two weeks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5351) JobTracker memory leak caused by CleanupQueue reopening FileSystem
[ https://issues.apache.org/jira/browse/MAPREDUCE-5351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated MAPREDUCE-5351: -- Attachment: MAPREDUCE-5351-1.patch > JobTracker memory leak caused by CleanupQueue reopening FileSystem > -- > > Key: MAPREDUCE-5351 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5351 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 1.1.2 >Reporter: Sandy Ryza >Assignee: Sandy Ryza >Priority: Critical > Attachments: MAPREDUCE-5351-1.patch, MAPREDUCE-5351.patch > > > When a job is completed, closeAllForUGI is called to close all the cached > FileSystems in the FileSystem cache. However, the CleanupQueue may run after > this occurs and call FileSystem.get() to delete the staging directory, adding > a FileSystem to the cache that will never be closed. > People on the user-list have reported this causing their JobTrackers to OOME > every two weeks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5351) JobTracker memory leak caused by CleanupQueue reopening FileSystem
[ https://issues.apache.org/jira/browse/MAPREDUCE-5351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated MAPREDUCE-5351: -- Attachment: MAPREDUCE-5351.patch > JobTracker memory leak caused by CleanupQueue reopening FileSystem > -- > > Key: MAPREDUCE-5351 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5351 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 1.1.2 >Reporter: Sandy Ryza >Assignee: Sandy Ryza >Priority: Critical > Attachments: MAPREDUCE-5351.patch > > > When a job is completed, closeAllForUGI is called to close all the cached > FileSystems in the FileSystem cache. However, the CleanupQueue may run after > this occurs and call FileSystem.get() to delete the staging directory, adding > a FileSystem to the cache that will never be closed. > People on the user-list have reported this causing their JobTrackers to OOME > every two weeks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
