[
https://issues.apache.org/jira/browse/HADOOP-1553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12514794
]
Owen O'Malley commented on HADOOP-1553:
---------------------------------------
Ok, I tried moving the logging down into the Task and that fixed the
performance problem, but doesn't help any code that writes directly to stdout
or stderr. That doesn't help.
So I propose running all Tasks via /bin/sh, using the shell to do the redirect
of stdin, stdout, and stderr. This will allow us to keep the system performant
and log the user's output.
> Extensive logging of C++ application can slow down task by an order of
> magnitude
> --------------------------------------------------------------------------------
>
> Key: HADOOP-1553
> URL: https://issues.apache.org/jira/browse/HADOOP-1553
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.13.0
> Reporter: Christian Kunz
> Assignee: Owen O'Malley
> Priority: Blocker
> Fix For: 0.14.0
>
>
> We observed that extensive logging (due to some configuration mistake) of a
> c++ application using the pipes interface can slow down the task by an order
> of magnitude. During that time disk usage was not high, with no abnormal
> memory usage, and basically idle CPU.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
