github-actions[bot] opened a new pull request, #76223: URL: https://github.com/apache/airflow/pull/76223
* Keep draining LocalExecutor results while a workload put is blocked Draining the result queue once before each put did not remove the deadlock reported in #70526. A worker can write a result between the drain and the put; if the activity pipe is full at that point the put blocks, and the worker keeps writing results until the result pipe is full too, so each side waits on the other. Linux pipes free space only a page at a time, which is why the regression test from #70545 hangs on loaded CI runners but not on macOS. Make the put on a short-lived helper thread and keep reading results on the main thread until it completes, so the result pipe can never fill up behind a blocked put. The workload is registered as running before it is sent, so a "running" result read while the put is in flight is not dropped. Generated-by: Claude Opus 5 * Undo a failed LocalExecutor put and spawn workers while one is blocked Registering a workload before its put meant a put that raised left the key running and counted although nothing was sent; restore the queued workload and its bookkeeping instead. Counting each workload before its put keeps a worker from decrementing the outstanding count below zero when a batch is interrupted part way. A blocked put can only complete once a live worker reads the activity pipe, but workers were only spawned or replaced after the batch, so with the spawn start method (or once every worker had died) a batch larger than the pipe waited forever. Spawn them from the drain loop whenever fewer than parallelism are alive: spawn starts one per call, so a single busy worker would otherwise hold the put until its task finished. Generated-by: Claude Opus 5 * Cover undoing a failed put while an earlier dispatch of the key is running The test only reached the no-earlier-dispatch path, so the remaining > 0 branch of _undo_dispatch had no coverage. (cherry picked from commit 1d1d17bb718dbb1362dbfabcefc37755c4074763) Co-authored-by: Jarek Potiuk <[email protected]> Generated-by: Claude Opus 5 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
