Siyao Meng created HDDS-16954:
---------------------------------
Summary: SnapshotChainManager iterator throws
NoSuchElementException when a snapshot is purged during iteration
Key: HDDS-16954
URL: https://issues.apache.org/jira/browse/HDDS-16954
Project: Apache Ozone
Issue Type: Bug
Components: Ozone Manager
Reporter: Siyao Meng
Assignee: Siyao Meng
{{SnapshotChainManager#iterator(boolean)}} is not safe against a concurrent
snapshot purge. {{hasNext()}} and {{next()}} read {{currentSnapshotId}} and
then call {{hasPreviousGlobalSnapshot}} / {{previousGlobalSnapshot}} (or the
next variants), which throw {{NoSuchElementException}} if the snapshot was
removed from the chain in between. The iterator does not hold the chain lock
across iterations, so callers such as {{DirectoryDeletingService#getTasks}},
{{KeyDeletingService#getTasks}}, {{SnapshotDeletingService}} and
{{SnapshotDefragService}} can hit this when a snapshot purge runs concurrently.
{code}
ERROR om.SnapshotChainManager
(SnapshotChainManager.java:hasPreviousGlobalSnapshot(516)) - No snapshot found
in snapshot chain for provided snapshotId: b0ef2d42-...
{code}
HDDS-16949 stops a {{getTasks()}} failure from cancelling the background
service schedule, but each affected service still loses a whole cycle (the AOS
task and every snapshot task) when the race hits.
h3. Fix
Make the iterator race safe, for example by taking a consistent snapshot of the
chain order under the chain lock when the iterator is created, or by ending the
walk gracefully when the current snapshot has been removed. Add a unit test
that purges a snapshot while iterating.
- Found in
https://github.com/apache/ozone/actions/runs/38252733251/job/114829762795
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
