SEPURI-SAI-KRISHNA commented on issue #13316:
URL: https://github.com/apache/seatunnel/issues/13316#issuecomment-7031388761

   I'd like to work on this one, if nobody else has started it.
   
   The chain is unchanged on current dev (`e2597998a9`): the callback in 
`JobMaster.initStateFuture` calls `cleanJob()` and only then 
`jobMasterCompleteFuture.complete(jobResult)`, `withTryCatch` swallows a throw 
from `cleanJob()`, and `run()` then blocks on `jobMasterCompleteFuture.join()`. 
A throw from `clearCheckpointIfNeed` also skips `storeJobInfo`, 
`storeFinishedJobState` and `scheduleRemoveJobStateMaps`, so the job state maps 
stay behind as well.
   
   Plan:
   - complete `jobMasterCompleteFuture` in a `finally`, so a cleanup failure 
can no longer hang `run()` or the clients;
   - run each cleanup step on its own and log a failure with the job id, so one 
failing step does not skip the others;
   - a `JobMasterTest` case on the real `initStateFuture()` and `cleanJob()`: 
with each step throwing in turn, the future completes with the job result and 
the other steps still run. It fails on dev.
   
   No change to the job result, the checkpoint or history formats, or any 
option.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to