[ https://issues.apache.org/jira/browse/MAPREDUCE-1783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12884367#action_12884367 ]
Hadoop QA commented on MAPREDUCE-1783: -------------------------------------- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12445178/submit-mapreduce-1783.patch against trunk revision 959509. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/279/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/279/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/279/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/279/console This message is automatically generated. > Task Initialization should be delayed till when a job can be run > ---------------------------------------------------------------- > > Key: MAPREDUCE-1783 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1783 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/fair-share > Affects Versions: 0.20.1 > Reporter: Ramkumar Vadali > Fix For: 0.22.0 > > Attachments: 0001-Pool-aware-job-initialization.patch, > 0001-Pool-aware-job-initialization.patch.1, submit-mapreduce-1783.patch > > > The FairScheduler task scheduler uses PoolManager to impose limits on the > number of jobs that can be running at a given time. However, jobs that are > submitted are initiaiized immediately by EagerTaskInitializationListener by > calling JobInProgress.initTasks. This causes the job split file to be read > into memory. The split information is not needed until the number of running > jobs is less than the maximum specified. If the amount of split information > is large, this leads to unnecessary memory pressure on the Job Tracker. > To ease memory pressure, FairScheduler can use another implementation of > JobInProgressListener that is aware of PoolManager limits and can delay task > initialization until the number of running jobs is below the maximum. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.