devmadhuu commented on code in PR #12062:
URL: https://github.com/apache/ozone/pull/12062#discussion_r4630627078
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerMetrics.java:
##########
@@ -34,6 +34,9 @@ public final class ContainerBalancerMetrics {
private final MetricsSystem ms;
+ @Metric("Number of times scheduling reached the shared iteration byte limit")
+ private MutableCounterLong numIterationLimitsSkipped;
Review Comment:
Okay. Renamed.
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerTask.java:
##########
@@ -557,56 +560,45 @@ private boolean initializeIteration() {
this.totalNodesInCluster = datanodeUsageInfos.size();
- double clusterAvgUtilisation = calculateAvgUtilization(datanodeUsageInfos);
- if (LOG.isDebugEnabled()) {
- LOG.debug("Average utilization of the cluster is {}",
clusterAvgUtilisation);
+ // Node include/exclude filters have already been applied above.
+ ContainerBalancerClusterSnapshot analysis =
ContainerBalancerClusterAnalyzer.analyze(datanodeUsageInfos.stream()
+ .map(node ->
node.toProto(ClientVersion.DEFAULT_VERSION)).collect(Collectors.toList()),
+ config.getThresholdAsRatio(), Collections.emptySet(),
Collections.emptySet());
+ if (analysis.hasIncompleteStorageTypeUsage()) {
+ lastInitializationFailureDetail = "Incomplete storage-type usage
reports; cannot safely select typed moves";
+ LOG.warn(lastInitializationFailureDetail);
+ return false;
}
-
- double threshold = config.getThresholdAsRatio();
- // over utilized nodes have utilization(that is, used / capacity) greater
than upper limit
- this.upperLimit = ContainerBalancerClusterAnalyzer.computeUpperLimit(
- clusterAvgUtilisation, threshold);
- // under utilized nodes have utilization(that is, used / capacity) less
than lower limit
- this.lowerLimit = ContainerBalancerClusterAnalyzer.computeLowerLimit(
- clusterAvgUtilisation, threshold);
-
- if (LOG.isDebugEnabled()) {
- LOG.debug("Lower limit for utilization is {} and Upper limit for
utilization is {}", lowerLimit, upperLimit);
+ if (analysis.getStorageTypeSnapshots().isEmpty()) {
+ storageTypeStates.add(new StorageTypeBalancingState(null,
analysis.getLowerLimit(), analysis.getUpperLimit(),
+ datanodeUsageInfos, analysis.getSourceDatanodes(),
analysis.getTargetDatanodes()));
+ } else {
+ analysis.getStorageTypeSnapshots().forEach((type, snapshot) ->
+ storageTypeStates.add(new StorageTypeBalancingState(type,
snapshot.getLowerLimit(), snapshot.getUpperLimit(),
+ datanodeUsageInfos, snapshot.getSourceDatanodes(),
snapshot.getTargetDatanodes())));
}
-
- long totalOverUtilizedBytes = 0L, totalUnderUtilizedBytes = 0L;
- // find over and under utilized nodes
- for (DatanodeUsageInfo datanodeUsageInfo : datanodeUsageInfos) {
+ Set<DatanodeDetails> sources = new HashSet<>();
+ Set<DatanodeDetails> targets = new HashSet<>();
Review Comment:
Thanks for pointing out, but moves are already safe here: a storage type
without both sources and targets is skipped by the move loop. For the counts, I
kept the existing meaning: a datanode is unbalanced when its usage is outside
the threshold band. Making them depend on whether a matching target exists
would change numDatanodesUnbalanced and the stop path for existing single-type
(DISK-only) clusters. `TestContainerBalancerDatanodeNodeLimit` covers that
contract.
What do you think ?
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerSelectionCriteria.java:
##########
@@ -426,6 +458,32 @@ Set<ContainerID> getExcludeNotFoundContainers() {
return excludeContainersNotFound;
}
+ /**
+ * Whether the container's replica on {@code node} sits on {@code
storageType}.
+ * Unknown types and conflicting recorded/volume types cannot participate in
a typed move.
+ */
+ private static boolean isReplicaOnStorageType(Set<ContainerReplica> replicas,
+ DatanodeDetails node, StorageType storageType) {
+ for (ContainerReplica replica : replicas) {
+ if (replica.getDatanodeDetails().equals(node)) {
+ if (replica.getStorageType() != null && replica.getVolumeStorageType()
!= null
+ && replica.getStorageType() != replica.getVolumeStorageType()) {
Review Comment:
Agree, refactoring done.
##########
hadoop-hdds/docs/content/feature/ContainerBalancer.md:
##########
@@ -84,9 +84,29 @@ does not count as spare SSD space. Byte and datanode limits
remain shared across
Assessment warns when storage-type reports are incomplete, and estimation
rejects those
reports. When no storage-type breakdown is available, both commands use
overall usage.
-These commands report and estimate balancing work. The running balancer still
chooses
-moves using overall datanode usage; selecting moves separately for each
storage type is
-separate work.
+### Balancing by storage type
+
+When complete storage-type usage reports are available, the running balancer
selects
+sources and targets separately for each type. For example, a datanode at 80%
SSD usage
+can send an SSD replica to another datanode at 20% SSD usage, even when both
datanodes
+have the same overall utilization. Free DISK space is not a destination for
that SSD copy.
+The selected type is included in the copy command so the destination uses a
matching volume.
+
+Storage types take turns within each iteration, and the first type rotates
between
+iterations. The iteration byte limit, each datanode's incoming and outgoing
byte limits,
+and the distinct-datanode limit are shared across all types. A datanode that
sends SSD
+replicas and receives DISK replicas counts once toward the datanode limit.
+
+For an existing replica without a recorded storage type, the balancer uses its
reported
+volume type. It skips replicas whose type is unknown or whose recorded type
conflicts
Review Comment:
Rephrased.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]