jojochuang opened a new pull request, #10082:
URL: https://github.com/apache/ozone/pull/10082
## What changes were proposed in this pull request?
HDDS-15014. Speed up EC container decommission
Please describe your PR in detail:
1. SCM Replication Manager Enhancements:
- Threshold-Based Switching: EC containers on decommissioning nodes
now automatically switch from simple replication to reconstruction if the
source node's replication load exceeds a configurable threshold
(hdds.scm.replication.decommission.ec.reconstruction.threshold, default: 5).
- Feature Flag: Added
hdds.scm.replication.decommission.ec.reconstruction.enabled (default: true) to
explicitly toggle this behavior.
- Global Concurrency Limit: Implemented a cluster-wide cap
(hdds.scm.replication.decommission.concurrency, default: 100) to prevent
cluster-wide performance degradation during large-scale decommissions.
- Observability: Added the InflightDecommission gauge to
ReplicationManagerMetrics to track simultaneous decommissioning tasks.
2. Datanode Scheduling Enhancements:
- Disk-Aware Volume Selection: The ReplicationSupervisor now tracks
in-flight tasks per physical disk volume.
- Non-Busy Disk Prioritization: The ContainerImporter and
DownloadAndImportReplicator were updated to prioritize target volumes with zero
in-flight tasks, ensuring better I/O distribution and avoiding disk bottlenecks.
3. Stability and Infrastructure:
- Bug Fix: Fixed a series of Hugo documentation build errors caused
by invalid date formats in markdown front matter.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-15014
## How was this patch tested?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]