[ https://issues.apache.org/jira/browse/HDFS-15756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17321866#comment-17321866 ]
zhangxiping edited comment on HDFS-15756 at 4/15/21, 3:04 AM: -------------------------------------------------------------- -_- like : {code:java} //代码占位符 AbstractDelegationTokenSecretManager public synchronized long renewToken(Token<TokenIdent> token, String renewer) throws InvalidToken, IOException { ByteArrayInputStream buf = new ByteArrayInputStream(token.getIdentifier()); DataInputStream in = new DataInputStream(buf); TokenIdent id = createIdentifier(); id.readFields(in); LOG.info("Token renewal for identifier: " + formatTokenId(id) + "; total currentTokens " + currentTokens.size()); long now = Time.now(); ... ... if (getTokenInfo(id) == null && id.getIssueDate() + 60000 > now) { throw new StandbyException("Renewal request for unknown token " + formatTokenId(id)+",will Failover to other server and retry"); } if (getTokenInfo(id) == null) { throw new InvalidToken("Renewal request for unknown token " + formatTokenId(id)); } updateToken(id, info); return renewTime; } {code} was (Author: zhangxiping): like : {code:java} //代码占位符 AbstractDelegationTokenSecretManager public synchronized long renewToken(Token<TokenIdent> token, String renewer) throws InvalidToken, IOException { ByteArrayInputStream buf = new ByteArrayInputStream(token.getIdentifier()); DataInputStream in = new DataInputStream(buf); TokenIdent id = createIdentifier(); id.readFields(in); LOG.info("Token renewal for identifier: " + formatTokenId(id) + "; total currentTokens " + currentTokens.size()); long now = Time.now(); ... ... if (getTokenInfo(id) == null && id.getIssueDate() + 60000 > now) { throw new StandbyException("Renewal request for unknown token " + formatTokenId(id)+",will Failover to other server and retry"); } if (getTokenInfo(id) == null) { throw new InvalidToken("Renewal request for unknown token " + formatTokenId(id)); } updateToken(id, info); return renewTime; } {code} > RBF: Cannot get updated delegation token from zookeeper > ------------------------------------------------------- > > Key: HDFS-15756 > URL: https://issues.apache.org/jira/browse/HDFS-15756 > Project: Hadoop HDFS > Issue Type: Bug > Components: rbf > Affects Versions: 3.0.0 > Reporter: hbprotoss > Priority: Major > > Affected version: all version with rbf > When RBF work with spark 2.4 client mode, there will be a chance that token > is missing across different nodes in RBF cluster. The root cause is that > spark renew the token(via resource manager) immediately after got one, as > zookeeper don't have a strong consistency guarantee after an update in > cluster, zookeeper client may read a stale value in some followers not synced > with other nodes. > > We apply a patch in spark, but it is still the problem of RBF. Is it possible > for RBF to replace the delegation token store using some other > datasource(redis for example)? -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org