Gargi-jais11 commented on code in PR #436:
URL: https://github.com/apache/ozone-site/pull/436#discussion_r3273298263
##########
docs/05-administrator-guide/02-configuration/04-performance/11-storage-capacity-planning.md:
##########
@@ -0,0 +1,142 @@
+---
+sidebar_label: Storage Capacity Planning
+---
+
+# Storage Capacity Planning
+
+In Apache Ozone, managing disk capacity is critical for maintaining high
availability. Ozone uses a multi-layered approach to ensure that Datanodes do
not run out of physical space, which could lead to data corruption or service
hangs.
+
+For background on Datanode volumes and related settings, see
[Datanodes](../../../core-concepts/architecture/datanodes). Full property
descriptions are in the [configuration appendix](../appendix).
+
+## When is a Disk Considered "Full"?
+
+A disk (or "Volume" in Ozone terminology) is considered **Full** for new data
allocations and replication targets when the available physical space falls
below a specific threshold.
+
+This threshold is the sum of two "Reserved Space" configurations plus a safety
buffer for the operation itself:
+
+`Failure Threshold = (Volume Min Free Space) + (DU Reserved Space) +
(Operation Buffer)`
+
+If a volume's free space is less than this threshold, SCM will stop selecting
this volume for new containers or as a destination for re-replication
(including decommission and maintenance moves).
+
+## Reserved Space Concepts
+
+Ozone provides two types of reserved space to protect the system:
+
+### 1. Volume Min Free Space
+
+This is a safety margin managed by the Datanode to prevent the disk from ever
reaching 100% capacity. It is the most important setting for capacity planning.
+
+- **Purpose**: Protects the Datanode from OS-level disk-full errors.
+- **Configuration**:
+ - `hdds.datanode.volume.min.free.space`: A fixed value (e.g., `20GB`).
+ - `hdds.datanode.volume.min.free.space.percent`: A percentage of the volume
capacity (e.g., `0.02` for 2%).
+- **Logic**: Ozone uses the **maximum** of the fixed value and the percentage.
+
+### 2. DU Reserved Space
+
+This reserves space for non-Ozone files on the same disk (e.g., OS logs,
system tools, or other services).
+
+- **Purpose**: Prevents Ozone from consuming space that is intended for other
uses on the host.
+- **Configuration**:
+ - `hdds.datanode.dir.du.reserved`: Fixed bytes per volume (e.g.,
`/data1:500MB`).
+ - `hdds.datanode.dir.du.reserved.percent`: Percentage per volume (default is
`0.0001` or 0.01%).
+
+### 3. Operation Buffer (2x Container Size)
+
+When Ozone replicates a container (e.g., during decommissioning), it requires
space to hold the incoming data in a temporary directory before importing it.
+
+- **Requirement**: Ozone requires free space equal to **2x the Container
Size**.
+- **Example**: For a default 5 GB container, this is **10 GB**.
+
+## Configuration Parameters
+
+| Property | Default | Description |
+| :--- | :--- | :--- |
+| `hdds.datanode.volume.min.free.space` | `20GB` | Fixed minimum free space
per volume. |
+| `hdds.datanode.volume.min.free.space.percent` | `0.02` (2%) |
Percentage-based minimum free space. |
+| `hdds.datanode.dir.du.reserved` | (unset) | Fixed bytes reserved for
non-Ozone use. |
+| `ozone.scm.container.size` | `5GB` | The target size for containers. |
+| `hdds.datanode.storage.utilization.critical.threshold` | `0.95` (95%) |
Threshold at which SCM logs a "Critical" space warning. |
Review Comment:
We also mention about warning utilisation threshold :
```
<property>
<name>hdds.datanode.storage.utilization.warning.threshold</name>
<value>0.75</value>
<tag>OZONE, SCM, MANAGEMENT</tag>
<description>
If a datanode overall storage utilization exceeds more than this
value, a warning will be logged while processing the nodeReport in SCM.
</description>
</property>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
