SaketaChalamchala opened a new pull request, #12037: URL: https://github.com/apache/ozone/pull/12037
Found by TLA+ model checking and code review of the OM optimized snapshot diff paths under [HDDS-15926](https://issues.apache.org/jira/browse/HDDS-15926), on commit https://github.com/apache/ozone/commit/d3b75c75bf9ad475551202477c3a21a3f4b968b7. The patch is generated with Specula (Claude Opus 5.5). ## What changes were proposed in this pull request? Optimized full snapshot diff drops real changes once the from snapshot's `lastTransactionInfo` moves due to `SnapshotPurge` or deep cleaning, and it drops creates, renames, and modifies on a newer-to-older diff. In full diff mode, `SnapshotDiffManager` passed the from snapshot's `lastTransactionInfo` index to `FullDiffSequentialReader`. The reader keeps a to-side row only when `updateID` is above that index (missing, `0`, and `DEFAULT_OM_UPDATE_ID` stay candidates). Rows at or below the gate are stored as present markers, and `MergeJoinSnapDiffWriter` emits nothing for them. `lastTransactionInfo` is not stable: deep cleaning (`OMKeyPurgeRequest`, `OMDirectoriesPurgeRequestWithFSO`) and snapshot purge or move (`OMSnapshotPurgeRequest`, `OMSnapshotMoveUtils`) rewrite it. `createTransactionInfo` is set once when the snapshot is created and is not rewritten. The gate also assumed the from snapshot was the older one. On a diff from a newer snapshot to an older one, every to-side row is at or below the from snapshot's creation index, so the same comparison drops creates, renames, and modifies and leaves only deletes. * `resolveFullDiffUpdateIdGate` now uses the from snapshot's creation transaction index, and only when both snapshots have `createTransactionInfo` and the from index is strictly lower than the to index. A * reverse diff, equal indexes, or a snapshot with no creation transaction info disables the gate, and every to-side row is compared. There is no fallback to `lastTransactionInfo`. The delta-diff path is unchanged. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-17742 ## How was this patch tested? * `TestOptimizedSnapshotDiff#testForcedFullDiffAfterSnapshotPurge`: purge an older snapshot so the from snapshot's `lastTransactionInfo` moves past its `createTransactionInfo`, then a forced full diff still reports the rename and the create for ordered and unordered reports. * `TestOptimizedSnapshotDiff#testForcedFullDiffFromNewerToOlderSnapshot`: a forced full diff from a newer snapshot to an older one reports the reversed rename and the delete, for both report modes. * Existing `TestFullDiffSequentialReader` coverage of a supplied gate and of a disabled gate. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
