HesandaLiyanage opened a new pull request, #11262: URL: https://github.com/apache/ozone/pull/11262
## What changes were proposed in this pull request? During Recon delta sync, `ReconTaskControllerImpl.processTasks()` executes derived tasks (e.g. `ContainerKeyMapperTask`, `NSSummaryTask`), which write derived data to Recon's local RocksDB (`recon.db`). By default, RocksDB writes are not synced (`sync = false`) and remain in the OS page cache. Immediately following task success, `recordRunCompletion()` commits the task cursor (`lastUpdatedSeqNumber` and `lastTaskRunStatus = 0`) to Derby, which issues an `fsync`-durable commit. If a host power loss occurs in that window, the un-synced RocksDB WAL tail at sequence $N$ is lost while the durable Derby cursor survives at sequence $N$. On restart, `OzoneManagerServiceProviderImpl.start()` compares the task cursor with the delta cursor; because both are at sequence $N$, it skips reprocessing. The next delta sync starts from sequence $N$, permanently dropping the applied update from Recon's derived tables. This pull request introduces a write-time durability barrier: 1. Adds `syncReconDbLog()` in `ReconTaskControllerImpl` to sync the derived RocksDB WAL via `DBStore.flushLog(true)` before advancing and committing the task status cursor. 2. If `syncReconDbLog()` fails (e.g., `RocksDatabaseException`), leaves the cursor unadvanced, marks `lastTaskRunStatus` as `-1`, and registers the task in `failedTasks` for retry/reprocessing. 3. Adds unit test `testDerivedDbSyncedBeforeCursorAdvanceOnSuccess` to `TestReconTaskControllerImpl` asserting that `flushLog(true)` is called before the cursor advances and `flushLog(false)` is never called. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-16463 ## How was this patch tested? 1. Unit tests: ```bash mvn -pl :ozone-recon test -Dtest=TestReconTaskControllerImpl -DskipShade -DskipRecon -DskipDocs ``` All 22 tests passed (0 failures, 0 errors). 2. Checkstyle: ```bash ./hadoop-ozone/dev-support/checks/checkstyle.sh ``` 0 violations across all 58 modules. --- Generated-by: Antigravity (Gemini 3.8 Flash) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
