dabla commented on issue #45930:
URL: https://github.com/apache/airflow/issues/45930#issuecomment-2607352971
I've also encountered the same behaviour, as the connection is blocking the
thread or something and thus the SQLExecuteQueryOperator task only stops once
the connection returns back, at least that's what I experienced.
So to avoid that issue, what I've done when using the JdbcHook, was by
specifying a connection timeout on the JDBC connection, that way when the task
has to timeout, the connection will also (even though the query will continue
to run on the database) and thus not block the thread of the Airflow worker,
but this was a specific solution with JDBC connections.
Bellow the code:
```
hook = DbApiHook.get_hook(conn_id="conn_id")
with hook.get_conn() as conn:
with closing(conn.cursor()) as cur:
stmt = conn.jconn.createStatement()
if timeout:
stmt.setQueryTimeout(
int(timeout.total_seconds())
) # Set the timeout in seconds
stmt.executeQuery(sql) # Execute the SQL query
return fetch_one_handler(cur)
```
So maybe in the hook some kind of timeout should also be specified on the
connection being used, or if you want a more finegrained approach per operator,
the you could specify the connection timeout through the hook_params of the
SQLExecuteQueryOperator? Still, this al depends if the underlying connection
supports it of course.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
