[
https://issues.apache.org/jira/browse/MAPREDUCE-1640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853346#action_12853346
]
Sreekanth Ramakrishnan commented on MAPREDUCE-1640:
---------------------------------------------------
The problem in gist:
* In bash scripts all the commands are spawned using fork + exec. Similar to
system() syscall. The process hierarchy is as follows:
{noformat}
15772 pts/0 S 0:00 \_ /bin/bash ./myscript.sh
15773 pts/0 S 0:00 | \_ sleep 100000
{noformat}
* So when kill 15772 is sent, the signal is not delivered to child.
* So parent exits and sleep does not check if the parent is alive or not
continues doing its work as it is a long running process.
The problem is similar to what is mentioned and addressed in HADOOP-2721
For this problem all the node health script which is spawned should do:
{{setsid; exec(node_health_path)}} and then instead of Process.destory() we do
{{kill -pid}}, now the problem is that pid of the process is now passed on to
java, so it will change to {{setsid; echo $$ ;exec(node_health_path)}} and we
should read the input stream to get process id.
Or alternate solution to the problem is the node health script configured
manages its children :-)
> Node health feature fails to blacklist a node if the health check script
> times out in some cases
> ------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1640
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1640
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Reporter: Vinod K V
> Fix For: 0.22.0
>
>
> Node health check feature fails to blacklist a TT if health check script
> times out. Below are the values that were set:
> - mapred.healthChecker.interval=60000
> - mapred.healthChecker.script.timeout=6000
> And the script was:
> {code}
> #!/bin/bash
> echo "start"
> sleep 100000
> echo "end"
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
