br413 opened a new pull request, #70161: URL: https://github.com/apache/airflow/pull/70161
## Summary - Remove `os.fork()`-based task event emission in the OpenLineage listener - Run task lifecycle handlers inline instead of forking from multi-threaded worker processes - Use `forkserver` multiprocessing context for `ProcessPoolExecutor` on scheduler-side emissions ## Problem On Python 3.12+, the OpenLineage listener emits: ``` DeprecationWarning: This process is multi-threaded, use of fork() may lead to deadlocks in the child. ``` This came from `_fork_execute()` calling `os.fork()` during task lifecycle events, and from `ProcessPoolExecutor` defaulting to the `fork` start method on Linux. ## Test plan - [ ] CI provider tests for `openlineage` pass - [ ] `pytest providers/openlineage/tests/unit/openlineage/plugins/test_listener.py` passes Closes #47160 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
