[
https://issues.apache.org/jira/browse/HADOOP-4428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12641033#action_12641033
]
Hemanth Yamijala commented on HADOOP-4428:
------------------------------------------
In an offline discussion with Vivek, we found that this patch is not correct,
as it moves to the job in the next queue after initializing a job in the
waiting queue. In fact, there are two deeper issues:
- Currently the scheduler, which calls {{initTasks}} can trigger a set up task
to be run. This does not seem correct, because just like clean up tasks, setup
tasks should be done outside the control of the scheduler. This should be
addressed in the framework, possibly as part of HADOOP-4421 ?
- The capacity scheduler does not check for the status of a job to be running
before assigning tasks from it when it initializes jobs in the waiting queue.
The implicit assumption here was that after a call to {{initTasks}}, the job
would be runnable. However, changes to the framework to run setup tasks broke
this. Even if setup tasks are done independently of the scheduler, it is still
possible that the scheduler is looking at a job whose setup task is still not
complete, and hence by definition is not runnable.
The first point is more of a design issue, and hence not a blocker for 0.19, as
currently nothing breaks because of the approach.
The second point must be addressed in the capacity scheduler, which should
actually fix the problem raised in the bug report as well.
> Job Priorities are not handled properly
> ----------------------------------------
>
> Key: HADOOP-4428
> URL: https://issues.apache.org/jira/browse/HADOOP-4428
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Environment: Cluster: 106 TTs MapCapacity=212, ReduceCapacity=212
> Single Queue=default, User Limit=25, Priorities = Yes.
> Using hadoop branch 0.19 revision=705159
> Reporter: Karam Singh
> Assignee: Vinod K V
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4428-20081017.1.txt, HADOOP-4428-20081020.txt
>
>
> Job Priorities are not handled properly
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
