zhangshenghang opened a new issue, #13317: URL: https://github.com/apache/seatunnel/issues/13317
### What happened Terminal task-status events carry no attempt/execution identifier, while the delivery mechanism *guarantees* they can arrive after a restore. A late terminal event from the previous attempt then overwrites the new attempt's state and releases the new attempt's slot. Chain (dev @ 89f91a89af): 1. Every deployment gets a fresh executionId (`PhysicalVertex.getTaskGroupImmutableInformation`, `PhysicalVertex.java:339-353`, `flakeIdGenerator.newId()`), but `NotifyTaskStatusOperation` only carries `TaskGroupLocation` — the master cannot tell attempts apart. 2. `TaskExecutionService.notifyTaskStatusToMaster` retries terminal-status delivery **indefinitely**, and specifically retries through `JobRestoreInProgressException` until the master finishes restoring (`TaskExecutionService.java:760-815`). So an old attempt's terminal report is mechanically guaranteed to be deliverable *after* the restore completes. 3. On arrival, `JobMaster.updateTaskExecutionState` → `PhysicalVertex.updateStateByExecutionService` (`PhysicalVertex.java:575-584`) → `updateTaskState`: the end-state guard (`PhysicalVertex.java:410-413`) no longer protects, because the restore's `reset()` already put the vertex back to CREATED. 4. Consequences: a late FAILED/CANCELED fails the healthy new attempt (another cancel + restore round, burning retry budget); a late FINISHED corrupts the end-state accounting; and `JobMaster.updateTaskExecutionState` releases the task-group resource for **any** terminal event (`JobMaster.java:1383-1389`) — releasing the slot the *new* attempt just obtained, while its task keeps running. ### Trigger sequence Worker flaps (memberRemoved) → its tasks marked FAILED → pipeline restores and redeploys → worker rejoins → the old task truly ends and its (retried) terminal report lands on the new attempt. ### Impact Healthy restored pipelines killed by ghost events; retry budget consumed; running tasks with released slots (slot double-assignment risk). ### Suggested fix Carry the deployment's executionId (or an attempt epoch) on `NotifyTaskStatusOperation`/`TaskExecutionState` and drop events whose id does not match the current deployment; validate slot ownership (sequence) before releasing on a terminal event. ### SeaTunnel Version dev (89f91a89af), Zeta engine -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
