[ https://issues.apache.org/jira/browse/MAPREDUCE-1783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12869832#action_12869832 ]
Hadoop QA commented on MAPREDUCE-1783: -------------------------------------- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12445113/0001-Pool-aware-job-initialization.patch.1 against trunk revision 946833. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/541/console This message is automatically generated. > Task Initialization should be delayed till when a job can be run > ---------------------------------------------------------------- > > Key: MAPREDUCE-1783 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1783 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/fair-share > Affects Versions: 0.20.1 > Reporter: Ramkumar Vadali > Fix For: 0.22.0 > > Attachments: 0001-Pool-aware-job-initialization.patch, > 0001-Pool-aware-job-initialization.patch.1 > > > The FairScheduler task scheduler uses PoolManager to impose limits on the > number of jobs that can be running at a given time. However, jobs that are > submitted are initiaiized immediately by EagerTaskInitializationListener by > calling JobInProgress.initTasks. This causes the job split file to be read > into memory. The split information is not needed until the number of running > jobs is less than the maximum specified. If the amount of split information > is large, this leads to unnecessary memory pressure on the Job Tracker. > To ease memory pressure, FairScheduler can use another implementation of > JobInProgressListener that is aware of PoolManager limits and can delay task > initialization until the number of running jobs is below the maximum. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.