Wei-Chiu Chuang created HDDS-15091:
--------------------------------------
Summary: CapacityVolumeChoosingPolicy compares available bytes not
utilization
Key: HDDS-15091
URL: https://issues.apache.org/jira/browse/HDDS-15091
Project: Apache Ozone
Issue Type: Bug
Reporter: Wei-Chiu Chuang
According to ozone-default.xml
{noformat}
<property>
<name>hdds.datanode.volume.choosing.policy</name>
<value>org.apache.hadoop.ozone.container.common.volume.CapacityVolumeChoosingPolicy</value>
<tag>OZONE, CONTAINER, STORAGE, MANAGEMENT</tag>
<description>
The class name of the policy for choosing volumes in the list of
directories.
Defaults to
org.apache.hadoop.ozone.container.common.volume.CapacityVolumeChoosingPolicy.
This volume choosing policy randomly chooses two volumes with remaining
space and then picks the one with lower utilization.
</description>
</property> {noformat}
But actual implementation compares available amount of bytes of volumes.
{code:java}
long firstAvailable = firstVolume.getCurrentUsage().getAvailable()
- firstVolume.getCommittedBytes();
long secondAvailable = secondVolume.getCurrentUsage().getAvailable()
- secondVolume.getCommittedBytes();
selectedVolume = firstAvailable < secondAvailable ? secondVolume : firstVolume;
{code}
This is not a problem if volumes are of the same capacity. However, if the
datanode is heterogeneous and volumes are of different capacity (e.g. 24TB HDD
vs 2TB NVMe SSD) it will not work as intended, and low density volumes will
fill up sooner than the high density ones.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
