[
https://issues.apache.org/jira/browse/MAPREDUCE-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kumar Vavilapalli updated MAPREDUCE-3596:
-----------------------------------------------
Attachment: MAPREDUCE-3596-20120111.txt
It turns out MAPREDUCE-3530 didn't work. I could reproduce the bug in a
test-case.
Attached patch should fix this. Essentially I am cleaning up the data
structures a little late so that any outstanding updates from NM about new
containers can be filtered before sending them off to schedulers. Also added
one more check for containers belonging to finished applications from reaching
the scheduler.
I also added the test-case which fails without the patch and passes with.
> Sort benchmark got hang after completion of 99% map phase
> ---------------------------------------------------------
>
> Key: MAPREDUCE-3596
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3596
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster, mrv2
> Affects Versions: 0.23.0
> Reporter: Ravi Prakash
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3596-20120111.txt, logs.tar.bz2, logs.tar.bz2
>
>
> Courtesy [~vinaythota]
> {quote}
> Ran sort benchmark couple of times and every time the job got hang after
> completion 99% map phase. There are some map tasks failed. Also it's not
> scheduled some of the pending map tasks.
> Cluster size is 350 nodes.
> Build Details:
> ==============
> Compiled: Fri Dec 9 16:25:27 PST 2011 by someone from
> branches/branch-0.23/hadoop-common-project/hadoop-common
> ResourceManager version: revision 1212681 by someone source checksum
> on Fri Dec 9 16:52:07 PST 2011
> Hadoop version: revision 1212592 by someone Fri Dec 9 16:25:27 PST
> 2011
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
