zhangshenghang opened a new issue, #13325: URL: https://github.com/apache/seatunnel/issues/13325
### What happened The per-vertex cancel retry loop has no attempt cap and runs while holding two levels of monitors, so a live-but-unresponsive worker can block the pipeline's entire state machine — including a user's cancel request (dev @ 89f91a89af): 1. `PhysicalVertex.noticeTaskExecutionServiceCancel` (`PhysicalVertex.java:457-507`) retries `CancelTaskOperation` **forever** while the target member is alive (`while (!taskFuture.isDone() && executionAddress != null && member != null)`, `Thread.sleep(2000)` between failed attempts); `i` is only used for logging. 2. It runs inside `cancel()` → `stateProcess` (both `synchronized` on the PhysicalVertex), reached synchronously from `SubPlan.updatePipelineState → stateProcess` (both `synchronized` on the SubPlan, `SubPlan.java:421-473`). So while a worker is up but not answering (long GC, severe overload — each `invoke().get()` taking up to the invocation timeout), the thread holds the SubPlan monitor: every `updatePipelineState`, `cancelPipeline`, end-state callback and restore step for that pipeline queues behind it. A user cancelling the job (`PhysicalPlan.cancelJob` → `cancelPipeline`) cannot make progress until every invoke either succeeds or the member is declared gone. ### Impact Pipeline state machine (including cancellation) frozen for minutes; in pathological cases the job cannot be cancelled without killing the worker or master. ### Suggested fix Bound the retry attempts (then mark the vertex CANCELED locally, as the member-unavailable branch at `PhysicalVertex.java:500-506` already does), and/or move the cancel sends off the state-machine thread (async with a completion callback). ### SeaTunnel Version dev (89f91a89af), Zeta engine -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
