jason810496 opened a new pull request, #75707:
URL: https://github.com/apache/airflow/pull/75707
- closes: #75533
## Why
Stopping a task that a language SDK runs signalled the runtime's pid alone,
so a CLI the task had started was reparented to PID 1 and kept running. That
child also holds the runtime open, so the runtime itself survived until the
SIGKILL escalation `killed_task_cleanup_time` (60 s by default) later.
## How
Python tasks have not had this problem since #65738: the task runner gets
its own process group and the supervisor signals the group with `os.killpg`.
`_PopenActivitySubprocess` never opted in, so the same `killpg` path in
`_signal_subprocess` was never used for a language SDK task. It now does, which
covers every way the supervisor stops a task (a cleared task,
`execution_timeout`, repeated heartbeat failures, task-success overtime) for
the TypeScript, Java and executable runtimes alike.
```python
proc = subprocess.Popen(
[*command, f"--comm={...}", f"--logs={...}"],
...
process_group=0,
)
```
`Popen` returns only once the child has exec'd, so the group always exists
before `kill()` can run. The fork path in `WatchedSubprocess.start` needs a
parent-side `setpgid` mirror for that; this one does not.
## What
- `_PopenActivitySubprocess.start` launches the runtime with
`process_group=0` and passes `new_process_group=True`, so the supervisor
signals the group.
- A startup that fails before the supervisor takes over signals the group
too, through a new `_signal_process_tree` helper that falls back to the pid
when the group cannot be signalled.
- Tests, including a regression test where the runtime spawns a child:
without the change the child outlives the stop and the runtime exits with
`-SIGTERM` while the child keeps running.
Not covered: a runtime that exits on its own and leaves children behind. The
pid is reaped, so the group can no longer be resolved. Python tasks have the
same gap.
---
##### Was generative AI tooling used to co-author this PR?
- [x] Yes, with help of Claude Code Opus 5 following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
