[
https://issues.apache.org/jira/browse/HADOOP-5165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12673867#action_12673867
]
Vinod K V commented on HADOOP-5165:
-----------------------------------
Some observations:
- There are now six iterations(!!) over the list of all jobs for getting setup
and cleanup tasks. The code can be re-factored to get this count down to three,
for e.g., by combining JobInProgress.obtainJobCleanupTask with map and reduce
slots into a single loop.
- With this patch, even after the above refactoring, we go through the list of
jobs multiple times in all cases while without this patch, we do it only in the
worst case. Though this may quicken the launching of job-setup, job-cleanup and
task-cleanup tasks, it will affect all the heartbeats. I am just curious how
the grid-mix numbers have turned out with this patch.
> getSetupAndCleanupTasks should return multiple tasks in a heartbeat
> -------------------------------------------------------------------
>
> Key: HADOOP-5165
> URL: https://issues.apache.org/jira/browse/HADOOP-5165
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.20.0
> Reporter: Amareshwari Sriramadasu
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.20.0, 0.21.0
>
> Attachments: patch-5165-1.txt, patch-5165.txt
>
>
> getSetupAndCleanupTasks in JobTracker returns one task per heartbeat. With
> HADOOP-3136, schedulers give multiple tasks per heartbeat,
> getSetupAndCleanupTasks should also give multiple tasks per heartbeat.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
