[
https://issues.apache.org/jira/browse/HADOOP-4305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amareshwari Sriramadasu updated HADOOP-4305:
--------------------------------------------
Release Note: Added configuration property
"mapred.max.tasktracker.blacklists", to specify the number of blacklists for a
task tracker by various jobs after which the task tracker can be blacklisted
across all jobs, defaults to 4.
Status: Patch Available (was: Open)
test-patch result on trunk:
{noformat}
[exec]
[exec] -1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] -1 tests included. The patch doesn't appear to include any new
or modified tests.
[exec] Please justify why no tests are needed for
this patch.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning
messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number
of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs
warnings.
[exec]
[exec] +1 Eclipse classpath. The patch retains Eclipse classpath
integrity.
[exec]
{noformat}
All core and contrib tests passed on my machine.
It is very difficult to write junit test for this. Did manual tests.
Manual tests include:
# Verified that a tracker should get blacklisted across all jobs iff
** #blacklists exceeds mapred.max.tracker.blacklists.
** #blacklists is 50% above the average #blacklists, over the active and
potentially faulty trackers
** 50% of the cluster is not blacklisted yet
# Verified Restarting the tracker makes it a healthy tracker. Verified
restart on a healthy tracker, blacklisted tracker and a lost tracker.
# Verified that Lost task tracker that bounced back should accept tasks again
# Verified that Once blacklisted tracker, if lost and comes back, it is not
healthy.
> repeatedly blacklisted tasktrackers should get declared dead
> ------------------------------------------------------------
>
> Key: HADOOP-4305
> URL: https://issues.apache.org/jira/browse/HADOOP-4305
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Christian Kunz
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.20.0
>
> Attachments: patch-4305-0.18.txt, patch-4305-1.txt, patch-4305-2.txt,
> patch-4305-3.txt
>
>
> When running a batch of jobs it often happens that the same tasktrackers are
> blacklisted again and again. This can slow job execution considerably, in
> particular, when tasks fail because of timeout.
> It would make sense to no longer assign any tasks to such tasktrackers and to
> declare them dead.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
