[
https://issues.apache.org/jira/browse/MAPREDUCE-2374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444239#comment-13444239
]
Andy Isaacson commented on MAPREDUCE-2374:
------------------------------------------
Arun, thanks for taking a look!
bq. Also, a simple test case to ensure we don't introduce "bash -c" in future
would have been helpful.
I'd be happy to add a testcase, but how would you suggest that we detect this?
Note that the ETXTBSY failure happens only on some systems, only under load,
and only if a script is written from the daemon and then executed by the shell.
There are other legitimate uses of "bash -c" where that's the only reasonable
way to do it, so we can't just prohibit the construct.
Thoughts?
> "Text File Busy" errors launching MR tasks
> ------------------------------------------
>
> Key: MAPREDUCE-2374
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2374
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Andy Isaacson
> Fix For: 1.2.0, 0.23.3, 2.2.0-alpha
>
> Attachments: failed_taskjvmsh.strace, mapreduce-2374-2.txt,
> mapreduce-2374-branch-1.patch, mapreduce-2374-on-20sec.txt,
> mapreduce-2374.txt, mapreduce-2374.txt, mapreduce-2374.txt,
> successfull_taskjvmsh.strace
>
>
> Some very small percentage of tasks fail with a "Text file busy" error.
> The following was the original diagnosis:
> {quote}
> Our use of PrintWriter in TaskController.writeCommand is unsafe, since that
> class swallows all IO exceptions. We're not currently checking for errors,
> which I'm seeing result in occasional task failures with the message "Text
> file busy" - assumedly because the close() call is failing silently for some
> reason.
> {quote}
> .. but turned out to be another issue as well (see below)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
