[ https://issues.apache.org/jira/browse/YARN-2962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526130#comment-14526130 ]
Vinod Kumar Vavilapalli commented on YARN-2962: ----------------------------------------------- bq. it will indeed be backwards incompatible. We should get this into trunk. To include this in branch-2, we should probably guard this with a config, and users will have to format the store to start using this. Otherwise, an RM restart will fail. Apologies for coming in late. I don't think we should give up on compatibility that easily. You can easily provide compat by doing one of the following - Forcing the new structure into its own parent directly. For e.g. instead of directly under RM_APP_ROOT, put it under something like RM_APP_ROOT/hierarchies, that way RM can read the old and new styles. /cc [~jianhe]. - If not that, we can simplify it without special hierarchies by having RM recursively read nodes all the time. As we already specifically look for "application_" prefix to read app-date, having application_1234_0456 right next to a 00/ directory is simply going to work without much complexity. That way, we should also have a more reasonable default instead of the current no-split default config value. > ZKRMStateStore: Limit the number of znodes under a znode > -------------------------------------------------------- > > Key: YARN-2962 > URL: https://issues.apache.org/jira/browse/YARN-2962 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager > Affects Versions: 2.6.0 > Reporter: Karthik Kambatla > Assignee: Varun Saxena > Priority: Critical > Attachments: YARN-2962.01.patch, YARN-2962.2.patch, YARN-2962.3.patch > > > We ran into this issue where we were hitting the default ZK server message > size configs, primarily because the message had too many znodes even though > they individually they were all small. -- This message was sent by Atlassian JIRA (v6.3.4#6332)