[
https://issues.apache.org/jira/browse/YARN-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siqi Li updated YARN-2366:
--------------------------
Attachment: YARN-2366.v1.patch
> Speed up history server startup time
> ------------------------------------
>
> Key: YARN-2366
> URL: https://issues.apache.org/jira/browse/YARN-2366
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 2.4.0
> Reporter: Siqi Li
> Assignee: Siqi Li
> Attachments: YARN-2366.v1.patch
>
>
> When history server starts up, It scans every history directories and put all
> history files into a cache, whereas this cache only stores 20K recent history
> files. Therefore, it is wasting a large portion of time loading old history
> files into the cache, and the startup time will keep increasing if we don't
> trim the number of history files. For example, when history server starts up
> with 2.5M history files in HDFS, it took ~5 minutes.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
