[ https://issues.apache.org/jira/browse/YARN-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shiwei Guo resolved YARN-4199. ------------------------------ Resolution: Duplicate > Minimize lock time in LeveldbTimelineStore.discardOldEntities > ------------------------------------------------------------- > > Key: YARN-4199 > URL: https://issues.apache.org/jira/browse/YARN-4199 > Project: Hadoop YARN > Issue Type: Improvement > Components: timelineserver, yarn > Reporter: Shiwei Guo > > In current implementation, LeveldbTimelineStore.discardOldEntities holds a > writeLock on deleteLock, which will block other put operation, which > eventually block the execution of YARN jobs(e.g. TEZ). When there is lots of > history jobs in timelinestore, the block time will be very long. In our > observation, it block all the TEZ jobs for several hours or longer. > The possible solutions are: > - Optimize leveldb configuration, so a full scan won't take long time. > - Take a snapshot of leveldb, and scan the snapshot, so we only need to hold > lock while getSnapshot. One question is that whether snapshot will take long > time or not, cause I have no experience with leveldb. -- This message was sent by Atlassian JIRA (v6.3.4#6332)