[
https://issues.apache.org/jira/browse/MAPREDUCE-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron Kimball updated MAPREDUCE-1499:
-------------------------------------
Attachment: MAPREDUCE-1499.patch
Attaching a patch to solve this problem. This locks the JT and then the
taskScheduler in that order before calls that will lock the JobInProgress, per
the discussion on MAPREDUCE-1436.
No new unit tests added because it's a deadlock issue.
Instead I tested manually by running two jobs in a row on a pseudo-distributed
cluster and inspecting the JobTracker state with JCarder. I did this with both
FIFO and fair scheduler. The issue reproduces without this patch; adding this
patch makes JCarder return "no cycles detected."
> JobTracker.finalizeJob inverts lock order and causes potential deadlock
> -----------------------------------------------------------------------
>
> Key: MAPREDUCE-1499
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1499
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.1, 0.20.2
> Reporter: Todd Lipcon
> Priority: Critical
> Attachments: jcarder_result_1.dot.png, jcarder_result_8.dot.png,
> MAPREDUCE-1499.patch
>
>
> This issue was brought up by Matei in MAPREDUCE-1436 as a fairsched bug, but
> it turns out it's a JT bug even with the fifo scheduler in unpatched 0.20.2.
> JobTracker.finalizeJob locks JT.jobs, JT.taskScheduler, etc, having gotten
> the JIP log before the JT lock.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
