[ https://issues.apache.org/jira/browse/MAPREDUCE-3568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3568: ----------------------------------------------- Attachment: MAPREDUCE-3568-20111220.txt Clean patch with simple test changes to validate the progress from the job directly (besides validating app progress at RM). > Optimize Job's progress calculations in MR AM > --------------------------------------------- > > Key: MAPREDUCE-3568 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3568 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mr-am, mrv2, performance > Affects Versions: 0.23.0 > Reporter: Vinod Kumar Vavilapalli > Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3568-20111215.1.txt, > MAPREDUCE-3568-20111220.txt > > > Besides catering to client requests, Job progress is calculated in every > heartbeat to the RM so as to print the MR AM's progress. Today the map and > reduce progresses are calculated by looking up of each task in a big map > while we can simply make do with a scan and aggregate. With large number of > tasks, this can make a difference. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira