[
https://issues.apache.org/jira/browse/HADOOP-1553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12517824
]
Arun C Murthy commented on HADOOP-1553:
---------------------------------------
+1 (assuming the shell shenanigans work on Windows and cgywin+cygpath... I only
tested the patch on Linux. *smile*)
Personally I'm glad we only have 3 files:
${HADOOP_LOG_DIR}/userlogs/${taskid}/[stdout|stderr|syslog] rather than
subdirs... much easier to debug stuff.
Super-minor nit: Given that {{mapred.userlog.retain.hours}} defaults to 24hrs
in hadoop-default.xml...
{noformat}
Index: src/java/org/apache/hadoop/mapred/TaskTracker.java
===================================================================
--- src/java/org/apache/hadoop/mapred/TaskTracker.java (revision 561588)
+++ src/java/org/apache/hadoop/mapred/TaskTracker.java (working copy)
@@ -1758,6 +1764,7 @@
Task task = umbilical.getTask(taskid);
JobConf job = new JobConf(task.getJobFile());
+ TaskLog.cleanup(job.getInt("mapred.userlog.retain.hours", 12));
task.setConf(job);
defaultConf.addFinalResource(new Path(task.getJobFile()));
{noformat}
should be:
{noformat}
+ TaskLog.cleanup(job.getInt("mapred.userlog.retain.hours", 24));
{noformat}
> Extensive logging of C++ application can slow down task by an order of
> magnitude
> --------------------------------------------------------------------------------
>
> Key: HADOOP-1553
> URL: https://issues.apache.org/jira/browse/HADOOP-1553
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.13.0
> Reporter: Christian Kunz
> Assignee: Owen O'Malley
> Priority: Blocker
> Fix For: 0.14.0
>
> Attachments: new-log-2.patch, new-log-3.patch
>
>
> We observed that extensive logging (due to some configuration mistake) of a
> c++ application using the pipes interface can slow down the task by an order
> of magnitude. During that time disk usage was not high, with no abnormal
> memory usage, and basically idle CPU.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
