[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16776199#comment-16776199 ]
He Xiaoqiao commented on HDFS-14305: ------------------------------------ Thanks [~csun],[~xkrogen] for your quick response. {quote}One potential issue with the patch 001 is that when keys are updated (which will call setSerialNo), it could go to a range that belongs to a different NameNode{quote} To [~csun], with patch 001 I think serial no will not be overlap for different namenodes if fixed number of namenodes in the same namespace. But it will appear when add/remove namenodes (e.g. observers) and have to re-config and restart all namenode in the same namespace. I think you also mean that, right? {quote}Instead of 1 bit, we can either pre-allocate a fixed number of bits (e.g., 5), or calculate the number of bits needed from the total number of configured namenodes. {quote} I agree with pre-allocate a fixed number of bits for different namenodes. [~xkrogen],[~csun] any more suggestions. > Serial number in BlockTokenSecretManager could overlap between different > namenodes > ---------------------------------------------------------------------------------- > > Key: HDFS-14305 > URL: https://issues.apache.org/jira/browse/HDFS-14305 > Project: Hadoop HDFS > Issue Type: Bug > Components: security > Reporter: Chao Sun > Assignee: Chao Sun > Priority: Major > Attachments: HDFS-14305.001.patch > > > Currently, a {{BlockTokenSecretManager}} starts with a random integer as the > initial serial number, and then use this formula to rotate it: > {code:java} > this.intRange = Integer.MAX_VALUE / numNNs; > this.nnRangeStart = intRange * nnIndex; > this.serialNo = (this.serialNo % intRange) + (nnRangeStart); > {code} > while {{numNNs}} is the total number of NameNodes in the cluster, and > {{nnIndex}} is the index of the current NameNode specified in the > configuration {{dfs.ha.namenodes.<nameservice>}}. > However, with this approach, different NameNode could have overlapping ranges > for serial number. For simplicity, let's assume {{Integer.MAX_VALUE}} is 100, > and we have 2 NameNodes {{nn1}} and {{nn2}} in configuration. Then the ranges > for these two are: > {code} > nn1 -> [-49, 49] > nn2 -> [1, 99] > {code} > This is because the initial serial number could be any negative integer. > Moreover, when the keys are updated, the serial number will again be updated > with the formula: > {code} > this.serialNo = (this.serialNo % intRange) + (nnRangeStart); > {code} > which means the new serial number could be updated to a range that belongs to > a different NameNode, thus increasing the chance of collision again. > When the collision happens, DataNodes could overwrite an existing key which > will cause clients to fail because of {{InvalidToken}} error. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org