amoghrajesh commented on issue #70398: URL: https://github.com/apache/airflow/issues/70398#issuecomment-5101793192
Thanks for the very thorough trace, yes this is a real bug, though it is one layer deeper than described here. The general defect in `_get_connection()` (sync mask_secret() calling `SUPERVISOR_COMMS.send()` from inside an already running event loop) was already fixed in task SDK by #55812 (`_async_get_connection()` / `_amask_connection_secrets()`), released in 3.1.0 and present in all 3.1.x releases. However, DBT cloud provider still has that exact pattern live on main today: - `DbtCloudHook.connection`) is a cached_property backed by the synchronous `get_connection()`. - `get_headers_tenants_from_connection()` is async and reads `self.connection` directly, with no `sync_to_async` wrapping so it runs on the triggerers own event loop thread. - `DbtCloudRunJobTrigger.run()` hits this on every poll: `get_job_status()` -> `get_job_details()` -> `await self.get_headers_tenants_from_connection()`. Notably, a related PR (#56791) already fixed the account id resolution in this same hook (`_resolve_account_id_async()` correctly uses `get_async_connection()`) but missed the connection headers path thats actually hit on every status poll. I'll put up a fix for this, but you will have to upgrade to Airflow 3.1 to really be getting this one fixed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
