[
https://issues.apache.org/jira/browse/HADOOP-5154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12674764#action_12674764
]
Matei Zaharia commented on HADOOP-5154:
---------------------------------------
Hi Hemanth,
The reason showJobs doesn't lock the JobTracker is because it only looks at
information in the fair scheduler. I have the lock around the JT for the other
two methods because update() will require a lock on the JT once preemption is
introduced in order to safely call killTask() on the JobTracker without causing
a deadlock. If you just lock the JT independently or the FS independently, you
can't cause a deadlock, but the problem is that the assignTasks method in the
JobTracker locks first the JT and then the FS and thus causes a problem.
However, maybe a better solution is to forget the lock on the JT in this patch,
and change the way killTasks is called so that it's not called with a lock
around the FairScheduler. This could be done by adding the tasks to kill into a
queue and having a second thread kill them. Since the preemption patch,
HADOOP-4665 is not going into release 0.20, I'm going to do that for this patch
and fix up the patch for HADOOP-4665 to ensure that you only need a lock on the
FS when calling update().
I can also turn off the refresh by default. How is it handled on the
JobTracker's web UI? For some reason I thought that refreshed.
> 4-way deadlock in FairShare scheduler
> -------------------------------------
>
> Key: HADOOP-5154
> URL: https://issues.apache.org/jira/browse/HADOOP-5154
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/fair-share
> Reporter: Vinod K V
> Assignee: Matei Zaharia
> Priority: Blocker
> Fix For: 0.18.4, 0.20.0
>
> Attachments: FairSchedulerDeadLock.txt, hadoop-5154-v0.patch,
> hadoop-5154-v1.patch, hadoop-5154-v2.patch
>
>
> This happened while trying to change the priority of a job from the scheduler
> servlet.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
