jojochuang commented on code in PR #11450:
URL: https://github.com/apache/ozone/pull/11450#discussion_r4236181702
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/ECUnderReplicationHandler.java:
##########
@@ -427,71 +474,88 @@ private int processDecommissioningIndexes(
throws IOException {
ContainerInfo container = replicaCount.getContainer();
Set<Integer> decomIndexes = replicaCount.decommissioningOnlyIndexes(true);
- int commandsSent = 0;
- if (!decomIndexes.isEmpty()) {
- LOG.debug("Processing decommissioning indexes {} for container {}.",
- decomIndexes, container.containerID());
- final List<DatanodeDetails> selectedDatanodes = getTargetDatanodes(
- container, decomIndexes.size(), usedNodes, excludedNodes);
+ if (decomIndexes.isEmpty()) {
+ return 0;
+ }
- ContainerPlacementStatus placementStatusWithSelectedTargets =
- validatePlacement(container, availableSourceNodes,
selectedDatanodes);
- if (!placementStatusWithSelectedTargets.isPolicySatisfied()) {
- LOG.debug("Target nodes + existing nodes for EC container {}" +
- " will not satisfy placement policy {}. Reason: {}. Selected" +
- " nodes: {}. Available source nodes: {}. Resuming recovery " +
- "regardless.",
- container.containerID(), containerPlacement.getClass().getName(),
- placementStatusWithSelectedTargets.misReplicatedReason(),
- selectedDatanodes, availableSourceNodes);
+ if
(replicationManager.getConfig().isEcDecommissionReconstructionEnabled()) {
+ for (Integer index : decomIndexes) {
+ Pair<ContainerReplica, NodeStatus> source = sources.get(index);
+ if (source != null && replicationManager.isNodeHighlyLoaded(
+ source.getLeft().getDatanodeDetails())) {
Review Comment:
Addressed in 5de75c0 (HDDS-15072). We skip the reconstruction switch when
`isReconstructionLimitReached()`, catch `CommandTargetOverloadedException` from
`sendThrottledReconstructionCommand`, and fall through to the existing 1-1
decommission copy path. Covered by
`testUnderReplicationWithDecomNodesFallsBackWhenLimitReached` and
`…WhenReconstructionThrottled`.
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/ECUnderReplicationHandler.java:
##########
@@ -427,71 +474,88 @@ private int processDecommissioningIndexes(
throws IOException {
ContainerInfo container = replicaCount.getContainer();
Set<Integer> decomIndexes = replicaCount.decommissioningOnlyIndexes(true);
- int commandsSent = 0;
- if (!decomIndexes.isEmpty()) {
- LOG.debug("Processing decommissioning indexes {} for container {}.",
- decomIndexes, container.containerID());
- final List<DatanodeDetails> selectedDatanodes = getTargetDatanodes(
- container, decomIndexes.size(), usedNodes, excludedNodes);
+ if (decomIndexes.isEmpty()) {
+ return 0;
+ }
- ContainerPlacementStatus placementStatusWithSelectedTargets =
- validatePlacement(container, availableSourceNodes,
selectedDatanodes);
- if (!placementStatusWithSelectedTargets.isPolicySatisfied()) {
- LOG.debug("Target nodes + existing nodes for EC container {}" +
- " will not satisfy placement policy {}. Reason: {}. Selected" +
- " nodes: {}. Available source nodes: {}. Resuming recovery " +
- "regardless.",
- container.containerID(), containerPlacement.getClass().getName(),
- placementStatusWithSelectedTargets.misReplicatedReason(),
- selectedDatanodes, availableSourceNodes);
+ if
(replicationManager.getConfig().isEcDecommissionReconstructionEnabled()) {
+ for (Integer index : decomIndexes) {
+ Pair<ContainerReplica, NodeStatus> source = sources.get(index);
+ if (source != null && replicationManager.isNodeHighlyLoaded(
+ source.getLeft().getDatanodeDetails())) {
+ LOG.info("Source node {} is highly loaded, switching to " +
+ "reconstruction for decommissioning container {}",
+ source.getLeft().getDatanodeDetails(), container.containerID());
+ return processReconstruction(replicaCount,
+ new ArrayList<>(decomIndexes), sources, availableSourceNodes,
+ excludedNodes, usedNodes);
Review Comment:
Addressed in 5de75c0. Added
`hasSufficientSourcesForDecommissionReconstruction()` requiring at least `data`
source indexes outside `decomIndexes` before switching; otherwise we use 1-1
copy. Test:
`testUnderReplicationWithDecomNodesFallsBackWhenInsufficientSources`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
