mobuchowski opened a new pull request, #65677: URL: https://github.com/apache/airflow/pull/65677
When the Airflow scheduler processes externally-changed task states (orphaned Celery TIs adopted after a scheduler restart, UI/API state changes routed through `process_executor_events → handle_failure`), the OpenLineage listener calls `os.fork()` in `_fork_execute` to emit the FAIL/COMPLETE event out-of-band. The forked child inherits the scheduler's SSL-wrapped Postgres connection pool and, because the AF3+ branch skipped `configure_orm(disable_connection_pool=True)` (guarded by `if not AIRFLOW_V_3_0_PLUS:` from #47580 to avoid crashing on the worker's `airflow-db-not-allowed:///` sentinel URL), the child issues DB queries over the same TLS socket as the parent, desynchronizing the OpenSSL sequence counter and crashing the scheduler's very next `session.flush()` with `psycopg2.OperationalError: SSL error: decryption failed or bad record mac`. This PR routes the scheduler-side "manual state change" emission through the existing `ProcessPoolExecutor` that DAG-run listeners already use (workers are initialized once via `_executor_initializer`, never share connections with the scheduler, and don't fork per event), and hardens the remaining `_fork_execute` path used by AF3 workers by gating `configure_orm` on `settings.engine is not None` so it rebuilds the ORM on the scheduler (where the engine is real) while staying a no-op on workers (where the sentinel URL would raise). ##### Was generative AI tooling used to co-author this PR? <!-- If generative AI tooling has been used in the process of authoring this PR, please change below checkbox to `[X]` followed by the name of the tool, uncomment the "Generated-by". --> - [ ] Yes (please specify the tool below) <!-- Generated-by: [Tool Name] following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) --> --- -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
