[
https://issues.apache.org/jira/browse/MAPREDUCE-1100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12852434#action_12852434
]
Ravi Gummadi commented on MAPREDUCE-1100:
-----------------------------------------
The patch existing now at MAPREDUCE-1057 doesn't solve the whole problem.
Currently, "tail -c" is done after task process has finished execution but
log.index file is created(and written to) by task itself. So log.index cannot
have correct index details of task logs(i.e. startingOffset and length of
stdout, stderr and syslog).
One way to solve this is to make TT write the index details to log.index file
once a task is done.
Thoughts ?
> User's task-logs filling up local disks on the TaskTrackers
> -----------------------------------------------------------
>
> Key: MAPREDUCE-1100
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1100
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Affects Versions: 0.21.0
> Reporter: Vinod K V
> Assignee: Vinod K V
> Attachments: MAPREDUCE-1100-20091102.txt,
> MAPREDUCE-1100-20091106.txt, MAPREDUCE-1100-20091216.2.txt,
> patch-1100-fix-ydist.2.txt
>
>
> Some user's jobs are filling up TT disks by outrageous logging.
> mapreduce.task.userlog.limit.kb is not enabled on the cluster. Disks are
> getting filled up before task-log cleanup via
> mapred.task.userlog.retain.hours can kick in.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
