DG47 commented on PR #74279:
URL: https://github.com/apache/airflow/pull/74279#issuecomment-6499616633
Thanks! Both are done in 190e1ce0ac.
**1. Trino and Oracle.** I ran the same check on both, with the same Airflow
3.3.2 setup and DAG as before:
- Trino 484 in Docker, trino provider 6.6.1, `trino` 0.339.0.
- Oracle AI Database 26ai Free (23.26), oracle provider 4.6.3, `oracledb`
26.0.0 in its default thin mode.
Both drivers return to Python while a statement runs, so `on_kill` runs
mid-statement. I checked on the server side with `system.runtime.queries` and
with `v$session` joined to `v$sql`:
| DB | kill | `cancel_on_kill=True` | `cancel_on_kill=False` |
|---|---|---|---|
| Trino | mark failed | cancelled through `cursor.cancel()`; query gone 0.3
s after SIGTERM (`USER_CANCELED`) | still `RUNNING` on Trino 60 s later |
| Trino | `execution_timeout` | gone 0.8 s after the timeout | also gone
after 0.8 s, because `trino`'s `Cursor.close()` calls `cancel()` and `run()`
closes the cursor while the timeout unwinds |
| Oracle | mark failed | cancelled through `connection.cancel()`; session no
longer active within 0.1 s of SIGTERM (`ORA-01013`) | stopped 5.2 s after
SIGTERM, when the supervisor SIGKILLed the task and Oracle noticed the dead
client |
| Oracle | `execution_timeout` | gone 0.5 s after the timeout | also gone
after 0.5 s, because closing the connection ends the session |
The times come from a 0.5 s poll of the server. psycopg 3 gave the same
results as before in the same run.
<details><summary>Task logs (mark failed)</summary>
```
# cancel_demo_trino / mark_failed
[14:22:24.694] INFO Executing: SELECT
'pr_trino_oracle_v2__trino/mark_failed' AS tag, count(*) FROM
tpch.sf1000.lineitem WHERE comment LIKE '%zz%q%'
[14:22:39.805] ERROR Server indicated the task shouldn't be running anymore.
Terminating process
[14:22:39.805] INFO Task instance is being killed, cancelling the running
SQL statement if there is one
[14:22:39.805] INFO Cancelling the running SQL statement through
cursor.cancel()
[14:22:39.811] ERROR Task failed with exception
TrinoUserError(type=USER_ERROR, name=USER_CANCELED,
message="Query was canceled", ...)
[14:22:39.816] ERROR Task killed!
# cancel_demo_oracle / mark_failed
[14:22:29.292] INFO Executing: SELECT
'pr_trino_oracle_v2__oracle/mark_failed' AS tag, count(*) FROM all_objects a,
all_objects b, all_objects c
[14:22:39.452] ERROR Server indicated the task shouldn't be running anymore.
Terminating process
[14:22:39.452] INFO Task instance is being killed, cancelling the running
SQL statement if there is one
[14:22:39.453] INFO Cancelling the running SQL statement through
connection.cancel()
[14:22:39.457] ERROR Task failed with exception
OperationalError: ORA-01013: User requested cancel of
current operation.
[14:22:39.469] ERROR Task killed!
```
</details>
I kept both drivers in the `cancel_query` docstring. It now says which
`cancel()` each one goes through: `cursor.cancel()` for `trino`, and
`connection.cancel()` for `psycopg` and for `oracledb` in thin mode. I didn't
test thick mode, which goes through the OCI client libraries. `operators.rst`
lists the same drivers.
The timeout rows also showed that `cancel_on_kill=False` doesn't always let
the statement finish. The cursor and the connection are still closed while the
timeout propagates, and these drivers stop the statement when that happens. The
`cancel_on_kill` docstring and `operators.rst` now say so.
**2.** `cancel_on_interrupt` is now a public attribute on `DbApiHook`. It's
documented in the class docstring and added to `sql.pyi`, and the operator sets
it from `cancel_on_kill`.
I'll do Snowflake together with the psycopg2 and SQLite follow-ups. One
question: `SnowflakeHook._run_command` would be calling `_track_running_cursor`
from another provider. Should I make that public here too, as
`track_running_cursor`, or is a private call fine for that follow-up?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
