ArafatKhan2198 opened a new pull request, #12000: URL: https://github.com/apache/ozone/pull/12000
## What changes were proposed in this pull request? `NSSummaryTask` updates Recon's namespace summary using three sub-tasks: FSO, Legacy and OBS. If the task fails, the controller retries it once on the same batch. Each sub-task reports a seek position, meaning how many events of the batch it has already saved, and on the retry it skips that many. That number is wrong in three cases, so the retry either applies events twice or skips them. **Example:** a batch has one Legacy key put and one OBS key put. 1. The Legacy sub-task saves its key and succeeds, but reports seek position 0. 2. The OBS sub-task fails. 3. On the retry, Legacy starts from 0 again and counts the same key a second time. 4. The retry succeeds, so no rebuild is triggered. The Legacy bucket now shows 2 files when it has 1. The other two cases: - After a threshold flush, the position is set to `eventCounter + 1`, which is one too far. The retry skips an event that was never saved. - An exception other than `IOException` escapes the loop. The position reached so far is lost, and the retry applies the saved events again. **Impact:** `/api/v1/namespace/summary` and `/du` show wrong file counts and sizes for the affected bucket until the next full rebuild. OM data is not affected. **How often:** rarely. A sub-task has to fail once and then succeed on the immediate retry. Most failures happen again on the retry, and that triggers a full rebuild that fixes the totals. One known trigger is a bucket being deleted and recreated with the same name exactly between the two attempts. **Change:** the seek position now always means "events of this batch already saved". In each of the three sub-tasks: - On success, return the end of the batch (`itrPos + eventCounter`). - After a threshold flush, set the position to `itrPos + eventCounter`. - Also catch `RuntimeException`, so the last saved position is returned. In FSO and Legacy, the threshold flush moves inside that `try`, the same way OBS already has it. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-17041 ## How was this patch tested? - Added `testRetryWithReturnedSeekPositionsDoesNotReapply` to `TestNSSummaryTask`. It processes a batch, then processes it again with the returned seek positions, and checks that the totals don't change. - Updated three existing assertions on the returned seek positions to the new meaning: one in `TestNSSummaryTask`, two in `TestNSSummaryTaskWithFSO`. - CI on the fork: https://github.com/ArafatKhan2198/ozone/actions/runs/42776143803 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
