saitejabandaru-in opened a new pull request, #71840: URL: https://github.com/apache/airflow/pull/71840
Closes #71525. ### Motivation When `DatabricksRunNowOperator(deferrable=True)` is used, the triggerer executes `DatabricksExecutionTrigger.run()`. Inside the async loop, the trigger polls for status via `DatabricksHook.a_get_run_state()`, which eventually calls `self._endpoint_url()`. That function tries to read `self.databricks_conn`, which is a `@cached_property` invoking `self.get_connection()`. In Airflow 3 (TaskSDK), `get_connection` bridges to a synchronous supervisor call using `async_to_sync`. Because this happens inside the triggerer's already-running asyncio loop, it crashes with `RuntimeError: You cannot use AsyncToSync in the same thread as an async event loop`. ### Changes - Added `adatabricks_conn` to `BaseDatabricksHook` which fetches the connection asynchronously via `aget_connection`. - Added `ahost`, `_a_endpoint_url`, and `_a_get_oidc_token_service_url` to prevent async methods from calling `self.host` (which also evaluates `databricks_conn` synchronously). - Refactored all `async def` methods in `BaseDatabricksHook` to use `(await self.adatabricks_conn())` and `await self._a_endpoint_url()`. - Updated unit tests to mock `adatabricks_conn`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
