[ https://issues.apache.org/jira/browse/YARN-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siqi Li reassigned YARN-2366: ----------------------------- Assignee: Siqi Li > Speed up history server startup time > ------------------------------------ > > Key: YARN-2366 > URL: https://issues.apache.org/jira/browse/YARN-2366 > Project: Hadoop YARN > Issue Type: Bug > Affects Versions: 2.4.0 > Reporter: Siqi Li > Assignee: Siqi Li > Attachments: YARN-2366.v1.patch > > > When history server starts up, It scans every history directories and put all > history files into a cache, whereas this cache only stores 20K recent history > files. Therefore, it is wasting a large portion of time loading old history > files into the cache, and the startup time will keep increasing if we don't > trim the number of history files. For example, when history server starts up > with 2.5M history files in HDFS, it took ~5 minutes. -- This message was sent by Atlassian JIRA (v6.2#6252)