Meng-Shuan Tsai created HDDS-16965:
--------------------------------------
Summary: Initialize disk balancer state from configured values
when diskBalancer.info is absent
Key: HDDS-16965
URL: https://issues.apache.org/jira/browse/HDDS-16965
Project: Apache Ozone
Issue Type: Bug
Reporter: Meng-Shuan Tsai
Assignee: Meng-Shuan Tsai
*Type:* Bug
*Component:* Datanode / DiskBalancer
*Problem*
When `diskBalancer.info` does not exist (first start of DiskBalancer on a
datanode), `DiskBalancerService.loadDiskBalancerInfo()` seeds persisted state
from `new DiskBalancerConfiguration()` instead of
`conf.getObject(DiskBalancerConfiguration.class)`. Only
`hdds.datanode.disk.balancer.should.run.default` is read from injected config.
As a result, five `ozone-site.xml` properties documented in [DiskBalancer
feature
doc|https://github.com/apache/ozone/blob/master/hadoop-hdds/docs/content/feature/DiskBalancer.md#diskbalancer-configurations]
are ignored on first init and written to `diskBalancer.info` as hard-coded
Java defaults:
* `hdds.datanode.disk.balancer.volume.density.threshold.percent` (default 10)
* `hdds.datanode.disk.balancer.max.disk.throughputInMBPerSec` (default 10)
* `hdds.datanode.disk.balancer.parallel.thread` (default 5)
* `hdds.datanode.disk.balancer.stop.after.disk.even` (default true)
* `hdds.datanode.disk.balancer.container.states` (default CLOSED,QUASI_CLOSED)
After the file is created, later restarts load from file; changing XML alone
does not fix the wrong values. `ozone admin datanode diskbalancer status`
reports the persisted (wrong) values with no warning.
*Root cause*
[DiskBalancerService.java#L243-L247|https://github.com/apache/ozone/blob/20e6a1c0f6039248373efd3eb1d7f2afcf7f1535/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/diskbalancer/DiskBalancerService.java#L243-L247]
*Steps to reproduce*
# Enable DiskBalancer on a datanode
(`hdds.datanode.disk.balancer.enabled=true`, default).
# Ensure `diskBalancer.info` is absent under the metadata / info dir.
# Set at least one non-default value among the five properties above in
`ozone-site.xml` (e.g. threshold 25, bandwidth 50).
# Start the datanode (or restart after deleting `diskBalancer.info` on an
unfixed build).
# Run `ozone admin datanode diskbalancer status <dn>`.
*Expected*
Status and `diskBalancer.info` reflect values from `ozone-site.xml`.
*Actual*
Status and YAML show Java defaults (10, 10, 5, true, CLOSED,QUASI_CLOSED).
Should-run default still follows XML.
*Impact*
Wrong bandwidth / parallelism / threshold / stop-after-even / movable states
when balancing runs; operator tuning via XML ineffective on new nodes until
admin `update`/`start` with explicit flags or manual file edit.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
