abdessalems opened a new issue, #11755: URL: https://github.com/apache/seatunnel/issues/11755
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Bug%22) and found no similar issues. ### What happened `TaskGroupLocation` is `{jobId, pipelineId, taskGroupId}` and is reused verbatim across pipeline restore generations. `TaskGroupExecutionTracker.taskDone()` removes the entry for that location from `executionContexts` unconditionally, so a late `taskDone()` belonging to an *earlier* generation can delete the execution context that the *current* generation's `deployLocalTask()` has just installed for the same location. There are only two sites in `TaskExecutionService` that write to that map: - `deployLocalTask()` installs the context (`executionContexts.put(location, ...)`) - `TaskGroupExecutionTracker.taskDone()` removes it (`executionContexts.remove(location)`) Because the key is identical across generations, the removal carries no notion of *which* generation it belongs to, so a stale tracker can evict a live entry. #11727 fixes the consequences of this race - every read of the map on the failure path is now null-safe, so a task group whose context disappears fails loudly instead of hanging the pipeline and the submitting `seatunnel.sh` client. It deliberately does **not** fix the race itself, which is what this issue tracks. ### What you expected to happen A `taskDone()` belonging to a previous restore generation should not be able to remove an execution context installed by a newer generation. ### How to reproduce The window is small and timing-dependent. It was originally observed via #11679, where `PaimonWithS3IT` hung for the full 180-minute CI timeout at roughly a 1-in-6 rate. `TaskDeployStaleContextRaceTest` (added in #11727) simulates the removal directly rather than waiting for the natural race, which is enough to cover the failure handling but does not reproduce the generation overlap itself. ### Possible direction Make the removal conditional on identity rather than key alone, so a tracker can only remove the context it installed - for example `executionContexts.remove(location, expectedContext)`, or carrying a generation/epoch on `TaskGroupContext` and comparing before eviction. Both need care: `taskDone()` runs from `BlockingWorker.run()`'s `finally` block, and `recycleClassLoader()` plus the `finishedExecutionContexts` handoff read the same entry immediately before and after the removal, so skipping the eviction has to leave those paths consistent rather than leaking contexts. ### Relevant context - #11679 - the hang this race produced - #11727 - fixes the consequences (all reads guarded, regression tests for both hangs) ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
