[
https://issues.apache.org/jira/browse/HDDS-16173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sreeja updated HDDS-16173:
--------------------------
Description:
We are adding three new CLI commands — assessment, dry-run, and recommend —
that all need the same cluster view. These commands run at client-side and
fetch datanode usage info via the existing getDatanodeUsageInfo RPC.
This Jira builds the shared foundation that is a common module that takes
datanode usage protos,
Responsibilities:
* Apply include/exclude filters on the proto list
* Compute cluster average utilization
* Compute upper/lower utilization limits from cluster average and threshold
* Compute each eligible node’s utilization (utilization = (capacity -
remaining) / capacity) and classify each as source, target, or balanced by
comparing with the upper/lower limits.
* Accumulate source/target counts and
totalOverUtilizedBytes/totalUnderUtilizedBytes
* Compute MaxUtilization, MinUtilization
* Compute cluster imbalance and bytes to move
* Build and return ContainerBalancerClusterSnapshot
The logic should match the way the running balancer uses in
ContainerBalancerTask.initializeIteration(), so CLI output and actual balancer
behaviour stay consistent.
Analyzer should return an object(ContainerBalancerClusterSnapshot) containing
totalEligibleDatanodes count, clusterAvgUtilization, clusterCapacityBytes,
MaxUtilization, MinUtilization, upperLimit, lowerLimit, sourceCount,
targetCount, totalOverUtilizedBytes, totalUnderUtilizedBytes, bytesToMove,
Imbalance (drift), top 5 sourceNodes hostnames, bottom 5 targetNodes hostnames.
We should also test this analysis.
was:
We are adding three new CLI commands — assessment, dry-run, and recommend —
that all need the same cluster view before any balancing runs. Those commands
run client-side and fetch datanode usage info via the existing
getDatanodeUsageInfo RPC.
This Jira builds the shared foundation that is a common module that takes
datanode usage protos,
Responsibilities:
* Apply include/exclude filters on the proto list
* Compute cluster average utilization
* Compute upper/lower utilization limits from cluster average and threshold
* Compute each eligible node’s utilization (utilization = (capacity -
remaining) / capacity) and classify each as source, target, or balanced by
comparing with the upper/lower limits.
* Accumulate source/target counts and
totalOverUtilizedBytes/totalUnderUtilizedBytes
* Compute MaxUtilization, MinUtilization
* Compute cluster imbalance and bytes to move
* Build and return ContainerBalancerClusterSnapshot
The logic should match what the running balancer uses in
ContainerBalancerTask.initializeIteration(), so CLI output and actual balancer
behavior stay consistent.
Analyzer should return an object(ContainerBalancerClusterSnapshot) containing
totalEligibleDatanodes count, clusterAvgUtilization, clusterCapacityBytes,
MaxUtilization, MinUtilization, upperLimit, lowerLimit, sourceCount,
targetCount, totalOverUtilizedBytes, totalUnderUtilizedBytes, bytesToMove,
Imbalance (drift), top 5 sourceNodes hostnames, bottom 5 targetNodes hostnames.
We should also test this analysis.
> Add cluster utilization analysis for container balancer CLI commands
> --------------------------------------------------------------------
>
> Key: HDDS-16173
> URL: https://issues.apache.org/jira/browse/HDDS-16173
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Sreeja
> Assignee: Sravani Revuri
> Priority: Major
>
> We are adding three new CLI commands — assessment, dry-run, and recommend —
> that all need the same cluster view. These commands run at client-side and
> fetch datanode usage info via the existing getDatanodeUsageInfo RPC.
>
> This Jira builds the shared foundation that is a common module that takes
> datanode usage protos,
> Responsibilities:
> * Apply include/exclude filters on the proto list
> * Compute cluster average utilization
> * Compute upper/lower utilization limits from cluster average and threshold
> * Compute each eligible node’s utilization (utilization = (capacity -
> remaining) / capacity) and classify each as source, target, or balanced by
> comparing with the upper/lower limits.
> * Accumulate source/target counts and
> totalOverUtilizedBytes/totalUnderUtilizedBytes
> * Compute MaxUtilization, MinUtilization
> * Compute cluster imbalance and bytes to move
> * Build and return ContainerBalancerClusterSnapshot
> The logic should match the way the running balancer uses in
> ContainerBalancerTask.initializeIteration(), so CLI output and actual
> balancer behaviour stay consistent.
> Analyzer should return an object(ContainerBalancerClusterSnapshot) containing
> totalEligibleDatanodes count, clusterAvgUtilization, clusterCapacityBytes,
> MaxUtilization, MinUtilization, upperLimit, lowerLimit, sourceCount,
> targetCount, totalOverUtilizedBytes, totalUnderUtilizedBytes, bytesToMove,
> Imbalance (drift), top 5 sourceNodes hostnames, bottom 5 targetNodes
> hostnames.
> We should also test this analysis.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
