SaketaChalamchala opened a new pull request, #12014:
URL: https://github.com/apache/ozone/pull/12014

   Found by TLA+ model checking and code review of the OM optimized snapshot 
diff paths under 
[HDDS-15926](https://issues.apache.org/jira/browse/HDDS-15926), on commit 
d3b75c75bf9ad475551202477c3a21a3f4b968b7. 
   The patch is generated with Specula (Claude Opus 5.5).
   
   Should be merged after https://github.com/apache/ozone/pull/12013
   
   ## What changes were proposed in this pull request?
   FSO snapshot diff dependency ordering (`SnapDiffDependencyGraph`) 
incorrectly applied a reoccupied-path skip when adding to-snapshot path-prefix 
edges. That skip broke ordering in two ways:
   
   1. Wrong “ordered” reports that do not replay — it dropped required edges 
for `RENAME`(target) under a reoccupied ancestor.
   **Example A: Wrong order (skip too aggressive for RENAME)**
   **Snapshot 1 → 2:**
   ```
   mkdir a
   create a/a
   snapshot 1
   mv a b
   mkdir a
   mv b/a a/a    # file ends at a/a in snapshot 2
   snapshot 2
   ```
   
   **Report entries:**
   ```
   RENAME a/a -> a/a
   RENAME a -> b
   CREATE a
   ```
   This leaves S2 with `b/a` instead of `a/a, `b`
   
   **After this patch:** the graph detects a cycle (true constraint conflict). 
With HDDS-17738, ordered jobs should fall back to an unordered report instead 
of `FAILED`.
   
   3. False cycles — it kept spurious edges from to-snapshot ancestor ops to 
`MODIFY` entries (and missed ancestor-freed paths), so jobs failed even when a 
valid order existed.
   **Example B: False cycle (skip too weak for MODIFY + nested recreate)**
   **Snapshot 1 → 2:**
   ```
   mkdir -p a/a
   create a/a/a
   snapshot 1
   mv a b
   mkdir -p a/a
   overwrite b/a/a          # MODIFY at from-snapshot path a/a/a
   snapshot 2
   ```
   
   **Report entries (valid replay order exists):**
   ```
   MODIFY a/a/a
   RENAME a -> b
   CREATE a
   CREATE a/a
   ```
   `MODIFY a/a/a` is on the old tree (path in snapshot 1). It must run before 
`RENAME a -> b`. It must not depend on `CREATE a`,  `/ CREATE a/a` on the new 
tree.
   
   The old logic still added (or failed to skip) a `CREATE a` → `MODIFY a/a/a` 
style edge when the path was freed via an ancestor rename (a, not a/a/a 
itself). That closed a false cycle → ordered job `FAILED`, while unordered 
worked.
   
   **After this patch:** no to-snapshot edges for `MODIFY`; 
   expected order: `MODIFY a/a/a, RENAME a -> b, CREATE a, CREATE a/a`
   
   
   The fix aligns edge rules with the path namespace contract: `MODIFY` never 
gets to-snapshot prefix edges, and `CREATE` / `RENAME(target)` always keep 
those edges. The reoccupied-path skip is removed (`buildReoccupiedPaths`, 
`shouldSkipReoccupiedPathToSnapshotEdge`).
   
   1. addPathPrefixToSnapshotEdges — skip `MODIFY` (and `DELETE`) entirely; 
only `CREATE` and `RENAME`(target) get to-snapshot prefix edges.
   2. Remove `buildReoccupiedPaths`, `shouldSkipReoccupiedPathToSnapshotEdge`, 
and `isStrictPathPrefix`.
   3. Always add `CREATE`/`RENAME(target) `ancestor → descendant edges (no 
skip).
   
   Rationale in one line: `MODIFY` is from-snapshot only for hierarchy; 
`CREATE`/`RENAME(target)` are to-snapshot only. Mixing `MODIFY` into 
to-snapshot prefix edges (with heuristics) caused both bugs.
   
   ## What is the link to the Apache JIRA
   
   https://issues.apache.org/jira/browse/HDDS-17739
   
   ## How was this patch tested?
   
    - Unit: `TestSnapDiffDependencyGraph`
         - New: 
`testRenameSourceReuseWithDescendantRenameIntoNewOccupierDetectsCycle`, 
`testRenameSourceReuseByNestedCreateWithDescendantModify`, 
`testDeleteAndRecreateNestedPathWithDescendantModifyAndRename`
         - Updated: `testDeleteAndRecreateWithOldTreeChildModifyAndRename` now 
expects a cycle (cross-parent rename into new occupier)
         - Existing reoccupied-path `MODIFY` cases updated / still pass with 
the new edge rules
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to