[
https://issues.apache.org/jira/browse/HADOOP-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12634434#action_12634434
]
Hemanth Yamijala commented on HADOOP-4053:
------------------------------------------
The default implementation for {{jobCompleted}} is calling {{jobRemoved}}. This
would result in {{jobRemoved}} being called twice - once when the job
completes, and once when it retires. Hence all implementations of
{{jobRemoved}} need to work correctly even if called twice. For e.g. in the
{{CapacityTaskScheduler}} we clear some data structures. I see that if called
twice, this may result in problems. For e.g.
{{qsi.numJobsByUser.get(job.getProfile().getUser());}} may return null when
called a second time, which is not being checked. I think it is safer to have
{{jobCompleted}} similar to other APIs - that is, as an abstract method and
implement it in schedulers. This way, we will be forced to check the
implementation is correct.
> Schedulers need to know when a job has completed
> ------------------------------------------------
>
> Key: HADOOP-4053
> URL: https://issues.apache.org/jira/browse/HADOOP-4053
> Project: Hadoop Core
> Issue Type: Improvement
> Affects Versions: 0.19.0
> Reporter: Vivek Ratan
> Assignee: Amar Kamat
> Fix For: 0.19.0
>
> Attachments: HADOOP-4053-v1.patch
>
>
> The JobInProgressListener interface is used by the framework to notify
> Schedulers of when jobs are added, removed, or updated. Right now, there is
> no way for the Scheduler to know that a job has completed. jobRemoved() is
> called when a job is retired, which can happen many hours after a job is
> actually completed. jobUpdated() is called when a job's priority is changed.
> We need to notify a listener when a job has completed (either successfully,
> or has failed or been killed).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
