[ https://issues.apache.org/jira/browse/MAPREDUCE-1648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Guilin Sun updated MAPREDUCE-1648: ---------------------------------- Status: Patch Available (was: Open) update for bugfix. > Use rolling to limit tasklogs > ----------------------------- > > Key: MAPREDUCE-1648 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1648 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: tasktracker > Reporter: Guilin Sun > Attachments: mapreduce-1648-design.pdf, MAPREDUCE-1648-v2.patch, > syslog-baidu-v2.patch > > Original Estimate: 0h > Remaining Estimate: 0h > > There are at least two types of task-logs: syslog and stdlog > Task-Jvm outputs syslog by log4j with TaskLogAppender, TaskLogAppender looks > just like "tail -c", it stores last N byte/line logs in memory(via queue), > and do real output only if all logs is commit and Appender is going to close. > The common problem of TaskLogAppender and 'tail -c' is keep everything in > memory and user can't see any log output while task is in progress. > So I'm going to try RollingFileAppender instead of TaskLogAppender, use > MaxFileSize&MaxBackupIndex to limit log file size. > RollingFileAppender is also suitable for stdout/stderr, just redirect > stdout/stderr to log4j via LoggingOutputStream, no client code have to be > changed, and RollingFileAppender seems better than 'tail -c' too. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.