Devesh Kumar Singh created HDDS-17818:
-----------------------------------------

             Summary: Storage policy observability: metrics, dashboard and 
compliance reporting
                 Key: HDDS-17818
                 URL: https://issues.apache.org/jira/browse/HDDS-17818
             Project: Apache Ozone
          Issue Type: Sub-task
          Components: Ozone Dashboards, Ozone Datanode, Ozone Filesystem, Ozone 
Manager, Ozone Recon
            Reporter: Devesh Kumar Singh
            Assignee: Devesh Kumar Singh


> Ozone shows how full SSD and DISK are, but not whether storage policies are 
> working. An administrator cannot see:
> - how much new HOT data is falling back from SSD to DISK
> - why SCM could not place data on a tier
> - whether enough SSD datanodes and pipelines are ready
> - whether repaired replicas or balancer moves stayed on the right medium
> - which buckets have data that doesn't match its policy
>
> Today no OM, SCM or datanode metric has a storage policy, tier or storage 
> type dimension. The only exception is the
> per-volume `StorageType` tag in `VolumeInfoMetrics`. SCM exports per-type 
> capacity only for DISK and SSD, not ARCHIVE.
>
> This JIRA tracks the work to make storage policy outcomes visible. Ozone 
> emits raw counters and gauges with bounded
> labels: policy, tier, storage type, replication and reason. Ratios such as 
> the fallback share are computed in
> PromQL, and a Grafana dashboard explains them. Per-bucket and per-key detail 
> stays in the CLI, not in Prometheus
> labels.
>
> Scope:
> - OM: committed bytes by policy, tier and fallback outcome.
> - SCM:
>   - placement failure reasons
>   - ARCHIVE capacity
>   - eligible datanodes per type
>   - pipeline readiness per tier
>   - repair outcome
>   - replica drift from the container tier
> - Datanode: Disk Balancer results per storage type.
> - A Storage Policy Grafana dashboard with example alerts.
> - A bucket and volume compliance summary in `ozone admin storagepolicy check`.
>
> Rules for all sub-tasks:
> - Keep existing aggregate metrics and names.
> - Add new series alongside them.
> - Report a missing tier or type as `UNKNOWN`, never as DISK.
> - Never use volume, bucket, key or exception text as a label.
>
> Related work tracked elsewhere:
> - Container Balancer per-type metrics: HDDS-17590, under HDDS-17583.
> - Per-key policy check: HDDS-17710.
> - Bucket mismatch listing: HDDS-17756.
> - Fallback contract: HDDS-17535.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to