[ https://issues.apache.org/jira/browse/HADOOP-16206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17498910#comment-17498910 ]
Duo Zhang commented on HADOOP-16206: ------------------------------------ After some investigating, ContainerLogAppender and ContainerRollingLogAppender are used for container logging. The goal is to limit the maximum size of logs. ContainerLogAppender will use a in memory queue to hold the tail of the log messages after exceeding the size limit, while ContainerRollingLogAppender is just a rolling file appender, we control the total size by setting the size limit for one file and the number of files to keep. So for these two appenders, I think maybe we could just leave the rolling file based one, as they have almost the same ability. And if we choose the rolling file based one, we do not need to implement a special appender any more, just use the RollingFileAppender for log4j2 is enough. We can read system properties in log4j2.properties so it is easy to control where the log. For TaskLogAppender, it is almost the same with ContainerLogAppender but I haven't seen where we plan to reference it. Searching the repo, I can only find references in 0.21.0's release note. https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/site/markdown/release/0.21.0/RELEASENOTES.0.21.0.md {noformat} MAPREDUCE-1086 | Major | hadoop commands in streaming tasks are trying to write to tasktracker's log This patch makes TT to set HADOOP_ROOT_LOGGER to INFO,TLA by default in the environment of taskjvm and its children. MAPREDUCE-1607 | Major | Task controller may not set permissions for a task cleanup attempt's log directory Fixed initialization of a task-cleanup attempt's log directory by setting correct permissions via task-controller. Added new log4j properties hadoop.tasklog.iscleanup and log4j.appender.TLA.isCleanup to conf/log4j.properties. Changed the userlogs for a task-cleanup attempt to go into its own directory instead of the original attempt directory. This is an incompatible change as old userlogs of cleanup attempt-dirs before this release will no longer be visible. {noformat} I think it provides the same ability with ContainerLogAppender before we have yarn. So I prefer we just remove it. For HttpRequestLogAppender, it is just a dummy appender, see the code here https://github.com/apache/hadoop/blob/5eab9719cbf6b9bddbdb4454a5f8e1ae12495492/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/HttpRequestLog.java#L86 We will replace it with the actual log writer at runtime. There is a related issue https://issues.apache.org/jira/browse/HADOOP-17526, where we want to replace it with Slf4jRequestLog, so we could just a normal log4j2 appender to configure it. So after HADOOP-17526, we could also remove it. For Log4jWarningErrorMetricsAppender, I think we need to port it to log4j2. Thanks. > Migrate from Log4j1 to Log4j2 > ----------------------------- > > Key: HADOOP-16206 > URL: https://issues.apache.org/jira/browse/HADOOP-16206 > Project: Hadoop Common > Issue Type: Sub-task > Affects Versions: 3.3.0 > Reporter: Akira Ajisaka > Assignee: Duo Zhang > Priority: Major > Attachments: HADOOP-16206-wip.001.patch > > > This sub-task is to remove log4j1 dependency and add log4j2 dependency. -- This message was sent by Atlassian Jira (v8.20.1#820001) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org