[
https://issues.apache.org/jira/browse/HDDS-16753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chi-Hsuan Huang updated HDDS-16753:
-----------------------------------
Description:
Follow-up to HDDS-16719.
{{OMMetrics.numSnapshotDeleted}} is incremented in {{OMSnapshotDeleteRequest}}
when a snapshot is marked {{SNAPSHOT_DELETED}}, but nothing decrements it when
{{SnapshotDeletingService}} purges that snapshot:
* {{OMSnapshotPurgeRequest}} removes the entry from {{snapshotInfoTable}} and
only calls {{omSnapshotIntMetrics.incNumSnapshotPurges()}}.
* {{decNumSnapshotDeleted()}} has had no callers since it was added in
HDDS-7506, and HDDS-16726 removes it as unused.
As a result, {{NumSnapshotDeleted}} keeps growing on a long-running OM and
reflects "snapshots deleted since startup" rather than "snapshots waiting to be
purged". The value is only corrected when {{updateActiveSnapshotMetrics()}}
recounts the table on OM start or checkpoint reload. The bundled OM Snapshot
dashboard plots this raw value, so the drift is visible there.
Proposed changes:
* Re-add {{decNumSnapshotDeleted()}} using {{decr()}} (the field is a gauge
after HDDS-16719) and call it in {{OMSnapshotPurgeRequest}} for each snapshot
actually purged, skipping the case where the snapshot is already gone from the
table.
* Assert {{NumSnapshotDeleted}} in {{TestOMSnapshotPurgeRequestAndResponse}}.
was:
Follow-up to HDDS-16719.
{{OMMetrics.numSnapshotDeleted}} is incremented in {{OMSnapshotDeleteRequest}}
when a snapshot is marked {{SNAPSHOT_DELETED}}, but nothing decrements it when
{{SnapshotDeletingService}} purges that snapshot:
* {{OMSnapshotPurgeRequest}} removes the entry from {{snapshotInfoTable}} and
only calls {{omSnapshotIntMetrics.incNumSnapshotPurges()}}.
* {{decNumSnapshotDeleted()}} has had no callers since it was added in
HDDS-7506.
As a result, {{NumSnapshotDeleted}} keeps growing on a long-running OM and
reflects "snapshots deleted since startup" rather than "snapshots waiting to be
purged". The value is only corrected when {{updateActiveSnapshotMetrics()}}
recounts the table on OM start or checkpoint reload. The bundled OM Snapshot
dashboard plots this raw value, so the drift is visible there.
Proposed changes:
* Call {{decNumSnapshotDeleted()}} in {{OMSnapshotPurgeRequest}} for each
snapshot actually purged, skipping the case where the snapshot is already gone
from the table.
* Assert {{NumSnapshotDeleted}} in {{TestOMSnapshotPurgeRequestAndResponse}}.
> NumSnapshotDeleted is not decremented when a snapshot is purged
> ---------------------------------------------------------------
>
> Key: HDDS-16753
> URL: https://issues.apache.org/jira/browse/HDDS-16753
> Project: Apache Ozone
> Issue Type: Bug
> Components: Ozone Manager
> Reporter: Chi-Hsuan Huang
> Assignee: Chi-Hsuan Huang
> Priority: Major
>
> Follow-up to HDDS-16719.
> {{OMMetrics.numSnapshotDeleted}} is incremented in
> {{OMSnapshotDeleteRequest}} when a snapshot is marked {{SNAPSHOT_DELETED}},
> but nothing decrements it when {{SnapshotDeletingService}} purges that
> snapshot:
> * {{OMSnapshotPurgeRequest}} removes the entry from {{snapshotInfoTable}} and
> only calls {{omSnapshotIntMetrics.incNumSnapshotPurges()}}.
> * {{decNumSnapshotDeleted()}} has had no callers since it was added in
> HDDS-7506, and HDDS-16726 removes it as unused.
> As a result, {{NumSnapshotDeleted}} keeps growing on a long-running OM and
> reflects "snapshots deleted since startup" rather than "snapshots waiting to
> be purged". The value is only corrected when
> {{updateActiveSnapshotMetrics()}} recounts the table on OM start or
> checkpoint reload. The bundled OM Snapshot dashboard plots this raw value, so
> the drift is visible there.
> Proposed changes:
> * Re-add {{decNumSnapshotDeleted()}} using {{decr()}} (the field is a gauge
> after HDDS-16719) and call it in {{OMSnapshotPurgeRequest}} for each snapshot
> actually purged, skipping the case where the snapshot is already gone from
> the table.
> * Assert {{NumSnapshotDeleted}} in {{TestOMSnapshotPurgeRequestAndResponse}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
