[
https://issues.apache.org/jira/browse/HDFS-15383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17246982#comment-17246982
]
Yuxuan Wang commented on HDFS-15383:
------------------------------------
Hi ~ [~fengnanli][~elgoiri][~hexiaoqiao]
After disable watcher, tokens in router memory can be stale. And client may
auth failed if the token is renewed but router don't rebuild cache yet.
Or there is some misunderstand in my mind? Plz figure out, Thx!
> RBF: Disable watch in ZKDelegationSecretManager for performance
> ---------------------------------------------------------------
>
> Key: HDFS-15383
> URL: https://issues.apache.org/jira/browse/HDFS-15383
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Fengnan Li
> Assignee: Fengnan Li
> Priority: Major
> Fix For: 3.4.0
>
>
> Based on the current design for delegation token in secure Router, the total
> number of watches for tokens is the product of number of routers and number
> of tokens, this is due to ZKDelegationTokenManager is using PathChildrenCache
> from curator, which automatically sets the watch and ZK will push the sync
> information to each router. There are some evaluations about the number of
> watches in Zookeeper has negative performance impact to Zookeeper server.
> In our practice when the number of watches exceeds 1.2 Million in a single ZK
> server there will be significant ZK performance degradation. Thus this ticket
> is to rewrite ZKDelegationTokenManagerImpl.java to explicitly disable the
> PathChildrenCache and have Routers sync periodically from Zookeeper. This has
> been working fine at the scale of 10 Routers with 2 million tokens.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
