[
https://issues.apache.org/jira/browse/MAPREDUCE-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12792829#action_12792829
]
Amar Kamat commented on MAPREDUCE-1316:
---------------------------------------
I think this scenario can be generalized further failed/killed jobs too.
Consider the following
1) a job is running and the jobtracker schedules some tasks
2) even before the tasktracker with the newly scheduled task reports back, the
job gets killed or is failed because of errors
3) jobtracker goes ahead and launches cleanup task (if any) which also
completes before the task-tracker with scheduled tasks report back
4) the job completes and later retires
5) the tracker with newly launched task reports back
The jobtracker simply ignores the updates as the job is retired. I think the
problem is in general with the tasks that are scheduled but the tasktacker
reports only after the job retires. Thoughts?
> JobTracker holds stale references to retired jobs via speculated tips
> ----------------------------------------------------------------------
>
> Key: MAPREDUCE-1316
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1316
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Amar Kamat
> Assignee: Amar Kamat
>
> JobTracker fails to remove speculative tasks' mapping from _taskToTIPMap_ if
> the job finishes and retires before the tracker (running the speculative
> tasks) reports back. In such cases a stale reference is held to
> TaskInProgress (and thus JobInProgress) long after the job is gone leading to
> memory leak.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
