[
https://issues.apache.org/jira/browse/HDDS-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDDS-17316:
----------------------------------
Labels: pull-request-available (was: )
> Cap the wait between retries when loading the initial secret key from SCM
> -------------------------------------------------------------------------
>
> Key: HDDS-17316
> URL: https://issues.apache.org/jira/browse/HDDS-17316
> Project: Apache Ozone
> Issue Type: Improvement
> Components: Security
> Reporter: Siyao Meng
> Assignee: Siyao Meng
> Priority: Major
> Labels: pull-request-available
>
> At startup, OM and Datanode load the current secret key from SCM in
> {{DefaultSecretKeySignerClient.loadInitialSecretKey}}. While SCM has not
> initialized secret keys yet ({{SECRET_KEY_NOT_INITIALIZED}}), it retries up
> to 100 times with exponential backoff that restarts every 10 retries. The
> base wait goes 2s, 4s, ..., 1024s, randomized by +/-50%, so a single wait can
> be up to about 25 minutes and the total about 5.7 hours. The code comment ("1
> 2 4 8 ... 512", "around 200min") understates both.
> So once SCM finishes initializing its keys, an OM or Datanode can stay stuck
> for many more minutes before trying again.
> Proposed fix: keep the exponential backoff but cap the base wait at 16s (2s,
> 4s, 8s, 16s, 16s, ...), so a single wait is at most 24s and the total for 100
> retries is around 26 minutes.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
