Pranaykarvi opened a new pull request, #64022:
URL: https://github.com/apache/airflow/pull/64022

   ## Problem
   
   When deadline callbacks run in the triggerer process, connection lookups via 
`Connection.async_get()` fail with `AirflowNotFoundException` for connections 
that exist in the Airflow metadata database (visible via `airflow connections 
get`).
   
   The root cause is in `ensure_secrets_backend_loaded()` — the triggerer 
process has no `SUPERVISOR_COMMS` and `_AIRFLOW_PROCESS_CONTEXT` is not set to 
`"server"`, so it falls into the fallback chain which only includes 
`EnvironmentVariablesBackend` and external backends. `MetastoreBackend` is 
never included, so UI-defined connections are not found.
   
   ## Fix
   
   Set `_AIRFLOW_PROCESS_CONTEXT=server` in `triggerer_run()` before starting 
the triggerer job runner. This makes the triggerer use the server secrets 
backend chain (which includes `MetastoreBackend`), the same as the scheduler 
and API server.
   
   Updated the `ensure_secrets_backend_loaded()` docstring to reflect that 
triggerer is now a server context.
   
   ## Testing
   
   Added `test_triggerer_server_context_includes_metastore_backend` verifying 
that `ensure_secrets_backend_loaded()` includes `MetastoreBackend` when 
`_AIRFLOW_PROCESS_CONTEXT=server` is set.
   
   Fixes #63929


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to