sundeep8967 opened a new pull request, #72286:
URL: https://github.com/apache/airflow/pull/72286

   <!--
   Thank you for contributing! Please make sure that your PR targets the `main` 
branch.
   -->
   
   Closes: #69277
   
   ### Description
   In `BigQueryAsyncHook`, `get_records` was implemented as a synchronous 
function rather than a native `async def` method. When deferrable operators or 
triggers processed large query results, the synchronous parsing blocked the 
Triggerer's `asyncio` event loop.
   
   This PR:
   1. Converts `BigQueryAsyncHook.get_records` in 
`providers/google/src/airflow/providers/google/cloud/hooks/bigquery.py` to 
`async def`, adding periodic yields via `await asyncio.sleep(0)` on large row 
batches.
   2. Updates all trigger callers in 
`providers/google/src/airflow/providers/google/cloud/triggers/bigquery.py` to 
`await hook.get_records(...)`.
   3. Updates unit tests in `test_bigquery.py` (hooks and triggers) to run 
asynchronously.
   
   Signed-off-by: Sundeep <[email protected]>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to