[ https://issues.apache.org/jira/browse/YARN-5955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721452#comment-15721452 ]
Varun Saxena commented on YARN-5955: ------------------------------------ Sorry was away so could not reply to either of the comments. Agree that Zookeeper is not necessarily meant to be a data store. There has been work going on in the community viz. YARN-5123 to support alternate state store implementations. The attempt at our end was to improve performance of RM recovery while using Zookeeper backed state store. And with the improvements done, it should serve use cases of most. Gradually, one more option community is moving towards is that we will use ATSv2 as a data store for completed apps and use RM state store only for running apps (which Zookeeper should be able to handle). ATSv2 though is still in alpha stage and requires a HBase cluster as well currently. So in the meanwhile we thought it will good to improve existing stable ZK based state store performance and allow it to store more data. And support use cases where having an HBase cluster (once ATSv2 stabilizes) is not a viable option. Async loading, using multiple threads and YARN-2962(i.e. store apps differently) are all an attempt in that direction. > Use threadpool or multiple thread to recover app > ------------------------------------------------ > > Key: YARN-5955 > URL: https://issues.apache.org/jira/browse/YARN-5955 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager > Affects Versions: 2.7.1 > Reporter: Zhaofei Meng > Assignee: Ajith S > Fix For: 2.7.1 > > > current app recovery is one by one,use thead pool can make recovery faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org