SakshamKapoor2911 opened a new pull request, #67881: URL: https://github.com/apache/airflow/pull/67881
closes: #67870 ### Description This PR fixes a deadlock in the `LocalExecutor` during executor shutdown (`end()`). Currently, the executor calls `proc.join()` on active worker processes. However, if a worker has written enough results to the `result_queue` to fill the OS-level pipe buffer (typically 64KB), the worker process blocks indefinitely on its `put()` call. Because the parent scheduler process is blocked on the unbounded `proc.join()` and not reading from `result_queue`, a classic multiprocessing deadlock occurs. The scheduler hangs indefinitely, stopping heartbeats and preventing systemd/Kubernetes from restarting the process (as it never exits). ### Changes 1. **Draining during Join:** Updated the shutdown loop in `LocalExecutor.end()` to continuously drain the `result_queue` using `_read_results()` while waiting for worker processes to join with a small timeout. 2. **Robust Queue Reading:** Wrapped the queue drainage loop in `_read_results()` with a `try/except (OSError, EOFError)` block to gracefully handle cases where pipes are already broken or closed during task exit. 3. **Forced Terminate:** Implemented the `LocalExecutor.terminate()` method to forcefully kill any remaining workers when a hard stop is requested. ### Context This contribution was created as part of the required capstone of the CodePath AI301 course, where students learn how to make responsible and effective use of generative AI tools for open-source software contributions. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (Antigravity IDE powered by Gemini) Generated-by: Antigravity IDE (Gemini) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) --- -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
