SaketaChalamchala opened a new pull request, #12037:
URL: https://github.com/apache/ozone/pull/12037

   Found by TLA+ model checking and code review of the OM optimized snapshot 
diff paths under 
[HDDS-15926](https://issues.apache.org/jira/browse/HDDS-15926), on commit 
https://github.com/apache/ozone/commit/d3b75c75bf9ad475551202477c3a21a3f4b968b7.
   The patch is generated with Specula (Claude Opus 5.5).
   
   ## What changes were proposed in this pull request?
   
   Optimized full snapshot diff drops real changes once the from snapshot's 
`lastTransactionInfo` moves due to `SnapshotPurge` or deep cleaning,  and it 
drops creates, renames, and modifies on a newer-to-older diff.
   
   In full diff mode, `SnapshotDiffManager` passed the from snapshot's 
`lastTransactionInfo` index to `FullDiffSequentialReader`. The reader keeps a 
to-side row only when `updateID` is above that index (missing, `0`, and 
`DEFAULT_OM_UPDATE_ID` stay candidates). Rows at or below the gate are stored 
as present markers, and `MergeJoinSnapDiffWriter` emits nothing for them. 
`lastTransactionInfo` is not stable: deep cleaning (`OMKeyPurgeRequest`, 
`OMDirectoriesPurgeRequestWithFSO`) and snapshot purge or move 
(`OMSnapshotPurgeRequest`, `OMSnapshotMoveUtils`) rewrite it. 
`createTransactionInfo` is set once when the snapshot is created and is not 
rewritten.
   
   The gate also assumed the from snapshot was the older one. On a diff from a 
newer snapshot to an older one, every to-side row is at or below the from 
snapshot's creation index, so the same comparison drops creates, renames, and 
modifies and leaves only deletes.
   
   * `resolveFullDiffUpdateIdGate` now uses the from snapshot's creation 
transaction index, and only when both snapshots have `createTransactionInfo` 
and the from index is strictly lower than the to index. A
   * reverse diff, equal indexes, or a snapshot with no creation transaction 
info disables the gate, and every to-side row is compared. There is no fallback 
to `lastTransactionInfo`. The delta-diff path is unchanged.
   
   ## What is the link to the Apache JIRA
   
   https://issues.apache.org/jira/browse/HDDS-17742
   
   ## How was this patch tested?
   
   * `TestOptimizedSnapshotDiff#testForcedFullDiffAfterSnapshotPurge`: purge an 
older snapshot so the from snapshot's `lastTransactionInfo` moves past its 
`createTransactionInfo`, then a forced full diff still reports the rename and 
the create for ordered and unordered reports.
   * `TestOptimizedSnapshotDiff#testForcedFullDiffFromNewerToOlderSnapshot`: a 
forced full diff from a newer snapshot to an older one reports the reversed 
rename and the delete, for both report modes.
   * Existing `TestFullDiffSequentialReader` coverage of a supplied gate and of 
a disabled gate.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to