SEPURI-SAI-KRISHNA commented on issue #13316: URL: https://github.com/apache/seatunnel/issues/13316#issuecomment-7031388761
I'd like to work on this one, if nobody else has started it. The chain is unchanged on current dev (`e2597998a9`): the callback in `JobMaster.initStateFuture` calls `cleanJob()` and only then `jobMasterCompleteFuture.complete(jobResult)`, `withTryCatch` swallows a throw from `cleanJob()`, and `run()` then blocks on `jobMasterCompleteFuture.join()`. A throw from `clearCheckpointIfNeed` also skips `storeJobInfo`, `storeFinishedJobState` and `scheduleRemoveJobStateMaps`, so the job state maps stay behind as well. Plan: - complete `jobMasterCompleteFuture` in a `finally`, so a cleanup failure can no longer hang `run()` or the clients; - run each cleanup step on its own and log a failure with the job id, so one failing step does not skip the others; - a `JobMasterTest` case on the real `initStateFuture()` and `cleanJob()`: with each step throwing in turn, the future completes with the job result and the other steps still run. It fails on dev. No change to the job result, the checkpoint or history formats, or any option. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
