potiuk opened a new pull request, #70562: URL: https://github.com/apache/airflow/pull/70562
`test_kill_terminates_whole_job_tree_under_job_control` failed on main in [this run](https://github.com/apache/airflow/actions/runs/30277364472/job/90021153407) with `job never wrote its pid file`. **Not a root-cause fix** — I could not reproduce it. This makes the next occurrence identify itself and restores the reruns meanwhile. The failure took 5.16s, i.e. the marker arrived and the whole 5s went on polling for the pid file. Two very different causes produce exactly that and the test can't tell them apart: `_run_bash_mc_under_pty` returns **silently** on EOF when the marker never arrives, so a launcher that died instantly looks identical to a job that died after launch. So: - the marker read is now asserted, quoting what the pty produced - the empty-pid assertion reports the job dir contents — the wrapper creates it before backgrounding anything, so missing means the launcher never got there, empty means the job was launched and died Also closes a real ordering hazard: `SUBMIT_DONE` only means the launcher returned, before `setsid` has forked and exec'd into the job. Closing the pty master hangs up the terminal, and `pty.fork()` makes the launcher the session leader, so hanging up in that window could take the job with it. The pty is now held open until the job records its own pid. This is hardening — see below. **What I tried:** macOS has no `setsid(1)` so the class skips there. Under Linux in Docker I ran the real wrapper through this harness 65 times — 25 idle, 40 at 0.35 CPU against six busy loops — and it recorded its pid every time, with and without the hold. The runner also had 88G free, so it isn't `set -euo pipefail` on a full disk. **Reruns:** `@pytest.mark.flaky(reruns=5)` returns. #69384 added it to the sibling test for an environment-dependent process-group race; #69490 dropped it while writing this variant. The first attempt's assertion text still reaches the CI log. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: Claude Code (Claude Opus 5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
