yandrey321 commented on code in PR #416: URL: https://github.com/apache/ozone-site/pull/416#discussion_r3237517845
########## docs/07-system-internals/02-data-operations/05-quota.md: ########## @@ -0,0 +1,90 @@ +--- +sidebar_label: Quota +--- + +# Implementation of Quota + +This page describes how Ozone enforces quotas in the Ozone Manager (OM). It complements user-facing quota documentation with an implementation-focused view. Tracked as [HDDS-15153](https://issues.apache.org/jira/browse/HDDS-15153). + +Ozone implements quota to manage physical data usage and the number of logical entities created by users. The implementation spans multiple layers, from volume-level restrictions to fine-grained bucket tracking and background garbage collection. + +## Types of quota + +Ozone tracks two dimensions of quota, stored on `OmVolumeArgs` and `OmBucketInfo`: + +- **Quota in bytes (space quota):** Limits the physical storage consumed by data blocks in the cluster, including replication. +- **Quota in namespace:** Limits how many logical objects exist: buckets under a volume, or keys and directories under a bucket. + +When a quota is unset or cleared, it uses `OzoneConsts.QUOTA_RESET`, which behaves as unbounded. + +## Volume-level quota enforcement + +At the volume layer, quota acts as a structural constraint across child buckets. The OM does not continuously re-sum active key sizes across the subtree to evaluate volume limits; volume space quota is enforced in terms of **per-bucket space quotas** and their aggregate. + +[`OMVolumeSetQuotaRequest`](https://github.com/apache/ozone/blob/master/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/volume/OMVolumeSetQuotaRequest.java) applies updates from a set-quota RPC. Bucket-level quotas that participate in volume accounting are reconciled through bucket metadata updates (including [`OMBucketSetPropertyRequest`](https://github.com/apache/ozone/blob/master/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/OMBucketSetPropertyRequest.java)). + +- **Namespace:** `checkQuotaNamespaceValid()` ensures the **current bucket count** in the volume does not exceed the new volume namespace quota. +- **Space:** `checkQuotaBytesValid()` requires that **every non-link bucket** in the volume has an explicit space quota set. It then ensures the **sum** of those bucket quotas does not exceed the volume’s space quota. + +## Bucket-level metrics and tracking + +At the bucket layer, `OmBucketInfo` records usage the OM uses for enforcement and reporting. Important fields include: + +| Field | Role | +| --- | --- | +| `usedBytes` | Active space for keys currently in the bucket namespace. | +| `usedNamespace` | Count of active keys and directories (per bucket semantics). | +| `snapshotUsedBytes` | Space still charged against quota while blocks are retained (for example pending physical delete or snapshot retention). Added as part of [HDDS-13756](https://issues.apache.org/jira/browse/HDDS-13756). | +| `snapshotUsedNamespace` | Namespace similarly retained until purge completes. | + +When OM checks remaining headroom for an operation, it uses **total** utilization by combining active and retained usage: + +```java +// OmBucketInfo (Ozone OM helpers) +public long getTotalBucketSpace() { + return usedBytes + snapshotUsedBytes; Review Comment: is there a way to check contribution of snapshotUsedBytes in total quota usage? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
