[ https://issues.apache.org/jira/browse/YARN-6521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15985230#comment-15985230 ]
Robert Kanter commented on YARN-6521: ------------------------------------- I think what [~piaoyu zhang] is asking is that when you disable log aggregation (because it was making too many small files and causing a problem for the NN), he still wants to be able to get logs from the YARN CLI logs command, which requires log aggregation. [~piaoyu zhang], instead of disabling log aggregation, you could periodically run the Archive Logs tool (https://hadoop.apache.org/docs/r2.8.0/hadoop-archive-logs/HadoopArchiveLogs.html), which combines all of the log files for an Application into one file. The YARN CLI logs command and JHS can still read these files. It does require Hadoop 2.8.0+ though. > Yarn Log-aggregation Transform Enable not to Spam the NameNode > -------------------------------------------------------------- > > Key: YARN-6521 > URL: https://issues.apache.org/jira/browse/YARN-6521 > Project: Hadoop YARN > Issue Type: Improvement > Reporter: zhangyubiao > > Nowdays We have large cluster,with the apps incr and container incr ,We split > an namespace to store applogs. > But the log-aggregation /tmp/app-logs incr also make the namespace responce > slow. > We want to chang yarn.log-aggregation-enable true -> false,But Transform the > the yarn log cli service also can get the app-logs. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org