Fengnan Li created HADOOP-16828:
-----------------------------------
Summary: Zookeeper Delegation Token Manager fetch sequence number
by batch
Key: HADOOP-16828
URL: https://issues.apache.org/jira/browse/HADOOP-16828
Project: Hadoop Common
Issue Type: Improvement
Reporter: Fengnan Li
Assignee: Fengnan Li
Attachments: Screen Shot 2020-01-25 at 2.25.06 PM.png, Screen Shot
2020-01-25 at 2.25.16 PM.png, Screen Shot 2020-01-25 at 2.25.24 PM.png
Currently in ZKDelegationTokenSecretManager.java the seq number is incremented
by 1 each time there is a request for creating new token. This will need to
send traffic to Zookeeper server. With multiple managers running, there is data
contention going on. Also, since the current logic of incrementing is using
tryAndSet which is optimistic concurrency control without locking. This data
contention is having performance degradation when the secret manager are under
volume of traffic.
The change here is to fetching this seq number by batch instead of 1, which
will reduce the traffic sent to ZK and make many operations inside ZK secret
manager's memory.
After putting this into production we saw huge improvement to the RPC
processing latency of get delegationtoken calls. Also, since ZK takes less
traffic in this way. Other write calls, like renew and cancel delegation tokens
are benefiting from this change.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
