[
https://issues.apache.org/jira/browse/HDDS-16181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sreeja Chintalapati updated HDDS-16181:
---------------------------------------
Description:
Add ozone admin containerbalancer recommend command
The command should:
# Fetch live datanode usage from SCM (via the existing getDatanodeUsageInfo
RPC)
# Read local Ozone configuration for global defaults (threshold, timeouts,
move limits etc.)
# Analyze the cluster using existing {{ContainerBalancerClusterAnalyzer}} /
{{ContainerBalancerClusterSnapshot}} (HDDS-16173)
# For each profile (SLOW, MEDIUM, FAST), produce:
** Recommended configuration parameters with a short rationale
** Time/iteration estimates by reusing
{{ContainerBalancerAdvisor.estimateDryRun()}} (HDDS-16179)
** Each recommended values should be validated first.
No --profile, config overrides, or interactive mode on this command
If the user wants custom inputs or refined estimates, they use dry-run instead.
But the command should allow the user to provide include/exclude datanodes and
threshold values.
{panel:title=Example output:}
RECOMMENDED CONFIGURATION (profile: SLOW)
══════════════════════════════════════════════════════
Parameter Value
Rationale
─────────────────────────────────────────────────────
--threshold <value>
<why this value>
--max-datanodes-percentage-to-involve <value> <why this
value>
--max-size-to-move-per-iteration-in-gb <value> <why
this value>
--max-size-entering-target-in-gb <value>
<why this value>
--max-size-leaving-source-in-gb <value>
<why this value>
--move-timeout-minutes <value>
<why this value>
--move-replication-timeout-minutes <value>
<why this value>
--balancing-iteration-interval-minutes <value>
<why this value>
--iterations <value>
<why this value>
─── Estimation ─────────────────────────────────────────────
Bytes to move: <cluster-derived>
Per iteration (estimate): <estimate>
Estimated iterations: <N>(planning estimate: <M>, includes
+30% buffer)
Estimated duration: upper bound <N> (planning estimate:
<M>, includes +30% buffer)
(assumes full move
timeout + interval each cycle)
(profile: MEDIUM)
...
(profile: FAST)
...
{panel}
was:
Add ozone admin containerbalancer recommend command which fetches datanode
usage info via the existing getDatanodeUsageInfo RPC. It should use the
cluster summary computed by the analyzer(HDDS-16173) to make the recommendation.
The command should show the recommended values for each config for each profile
i.e slow , medium and fast. Each recommended values should be validated first.
It should also show the estimates for each profile by re-using the estimate
computation logic fromHDDS-16179
To compute the recommended value for maxDatanodesPercentageToInvolvePerIteration
{code:java}
participatingNodes = max(2, 2 × max(sourceCount, targetCount))
rawPercentage = ceil(participatingNodes × 100.0 / totalEligibleDatanodes)
finalPercentage = min(rawPercentage, profile.cap)
{code}
Should also validate that :
allowedNodes = (int)(finalPercentage / 100.0 × totalEligibleDatanodes)
If allowedNodes < participatingNodes → bump percentage to make allowedNodes >=
participatingNodes
Else if allowedNodes < 2 → bump until allowedNodes >= 2
To compute the recommended value for maxSizeEnteringTarget,
maxSizeLeavingSource:
{code:java}
maxSizeEnteringTarget = min(profileDefaultEntering, maxSizeToMovePerIteration)
maxSizeLeavingSource = min(profileDefaultLeaving, maxSizeToMovePerIteration)
{code}
To compute the recommended value for maxSizeToMovePerIteration : re-use the
formula from HDDS-16179
For threshold , balancing interval and timeouts we can provide the default
config values itself.
The command should allow the user to provide include/exclude datanodes
> Add container balancer recommend CLI command to suggest config values for
> slow/medium/fast profiles
> ---------------------------------------------------------------------------------------------------
>
> Key: HDDS-16181
> URL: https://issues.apache.org/jira/browse/HDDS-16181
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Sreeja Chintalapati
> Assignee: Sreeja Chintalapati
> Priority: Major
>
> Add ozone admin containerbalancer recommend command
> The command should:
> # Fetch live datanode usage from SCM (via the existing getDatanodeUsageInfo
> RPC)
> # Read local Ozone configuration for global defaults (threshold, timeouts,
> move limits etc.)
> # Analyze the cluster using existing {{ContainerBalancerClusterAnalyzer}} /
> {{ContainerBalancerClusterSnapshot}} (HDDS-16173)
> # For each profile (SLOW, MEDIUM, FAST), produce:
> ** Recommended configuration parameters with a short rationale
> ** Time/iteration estimates by reusing
> {{ContainerBalancerAdvisor.estimateDryRun()}} (HDDS-16179)
> ** Each recommended values should be validated first.
> No --profile, config overrides, or interactive mode on this command
> If the user wants custom inputs or refined estimates, they use dry-run
> instead.
> But the command should allow the user to provide include/exclude datanodes
> and threshold values.
> {panel:title=Example output:}
> RECOMMENDED CONFIGURATION (profile: SLOW)
> ══════════════════════════════════════════════════════
> Parameter Value
> Rationale
> ─────────────────────────────────────────────────────
> --threshold <value>
> <why this value>
> --max-datanodes-percentage-to-involve <value> <why
> this value>
> --max-size-to-move-per-iteration-in-gb <value> <why
> this value>
> --max-size-entering-target-in-gb <value>
> <why this value>
> --max-size-leaving-source-in-gb <value>
> <why this value>
> --move-timeout-minutes <value>
> <why this value>
> --move-replication-timeout-minutes <value>
> <why this value>
> --balancing-iteration-interval-minutes <value>
> <why this value>
> --iterations <value>
> <why this value>
> ─── Estimation ─────────────────────────────────────────────
> Bytes to move: <cluster-derived>
> Per iteration (estimate): <estimate>
> Estimated iterations: <N>(planning estimate: <M>,
> includes +30% buffer)
> Estimated duration: upper bound <N> (planning estimate:
> <M>, includes +30% buffer)
> (assumes full move
> timeout + interval each cycle)
> (profile: MEDIUM)
> ...
> (profile: FAST)
> ...
> {panel}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
