devmadhuu commented on code in PR #11934:
URL: https://github.com/apache/ozone/pull/11934#discussion_r4509948018
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ReconTaskControllerImpl.java:
##########
@@ -487,6 +487,8 @@ private void processTasks(
// Track task delta processing failure
taskMetrics.incrTaskDeltaProcessingFailures(taskName);
+ // Retry it like a task that returned a failed result.
+ failedTasks.add(new
ReconOmTask.TaskResult.Builder().setTaskName(taskName).build());
Review Comment:
The PR description says that if a task throws, nothing from its batch was
saved. That assumption does not hold for `ContainerKeyMapperTask`.
During event processing, it:
1. Updates the global container count in SQL.
2. Saves the container mappings in key mapping rocks DB later, after
processing the batch.
Could we avoid assuming that an exception means the task has made no
persistent changes? `ContainerKeyMapperHelper.handlePutOMKeyEvent() `updates
the global container count in SQL (see
[here](https://github.com/apache/ozone/blob/2941b267ed6cae68eb5878e500d1aa092151d6b4/hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ContainerKeyMapperHelper.java#L359-L360))
before the batch’s RocksDB mappings are committed.
For example, a batch introduces containers A and B. A’s SQL count increment
succeeds, then processing B throws a transient unchecked SQL exception.
Retrying the entire batch counts A again because its mapping was never
committed. If the retry succeeds, the cursor advances with an inflated
container count and no rebuild is requested.
Could we either signal reinitialization directly for exceptions where
partial writes cannot be ruled out, or make those writes safe to repeat? Please
add a regression test that fails after the first count increment and verifies
the final count. The current mocked-task tests do not cover persistent side
effects.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]