[
https://issues.apache.org/jira/browse/YARN-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13803148#comment-13803148
]
Bikas Saha commented on YARN-1307:
----------------------------------
It is fine to store a batch of delegation token in the same znode. In fact that
may be better than creating a huge number of znode. A setData operation is easy
to do on a znode. So the stateStore can keep the current (upto 500kb) batch in
memory. When a new delegation token arrives then the store can add it to the
batch and overwrite the znode data with the new batch. Keep doing it till the
batch size is ~500kb. When the data size exceeds the size of a batch then a new
znode is created.
There is no need to encode the sequence number in the name. It can be made part
of the znode data. The sequence number znode stores the sequence number of the
tokens. We encoded the sequence number in the name for HDFS so that we dont
have to go to the data nodes to write the data. this is not necessary in ZK
since it supports a setData operation that overwrites the previous data. We can
simply name the znode something like RMDelegationTokenSequenceNumber.
> Rethink znode structure for RM HA
> ---------------------------------
>
> Key: YARN-1307
> URL: https://issues.apache.org/jira/browse/YARN-1307
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
> Reporter: Tsuyoshi OZAWA
> Assignee: Tsuyoshi OZAWA
> Attachments: YARN-1307.1.patch, YARN-1307.2.patch
>
>
> Rethink for znode structure for RM HA is proposed in some JIRAs(YARN-659,
> YARN-1222). The motivation of this JIRA is quoted from Bikas' comment in
> YARN-1222:
> {quote}
> We should move to creating a node hierarchy for apps such that all znodes for
> an app are stored under an app znode instead of the app root znode. This will
> help in removeApplication and also in scaling better on ZK. The earlier code
> was written this way to ensure create/delete happens under a root znode for
> fencing. But given that we have moved to multi-operations globally, this isnt
> required anymore.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
