[
https://issues.apache.org/jira/browse/YARN-6521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15985289#comment-15985289
]
Junping Du commented on YARN-6521:
----------------------------------
[~piaoyu zhang], I think what [~rkanter] mentioned above should at least
partially work for you. If that is still not enough, I am concerning disabling
log aggregation and allow some app's aggregated log on demand will bring many
other issues, like: how to manage local container logs and how to make sure
app's log still exists when log CLI runs, etc.
An alternative way is to set TTL to aggregated logs to keep latest
finished/running apps only, it can be flexible in policies, like: time based,
e.g. latest 15 days; app number based, e.g. latest 1000 apps or even size based
(considering long running apps).
CC [~xgong], [~jlowe].
> Yarn Log-aggregation Transform Enable not to Spam the NameNode
> --------------------------------------------------------------
>
> Key: YARN-6521
> URL: https://issues.apache.org/jira/browse/YARN-6521
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: zhangyubiao
>
> Nowdays We have large cluster,with the apps incr and container incr ,We split
> an namespace to store applogs.
> But the log-aggregation /tmp/app-logs incr also make the namespace responce
> slow.
> We want to chang yarn.log-aggregation-enable true -> false,But Transform the
> the yarn log cli service also can get the app-logs.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
