This is quite useful! Nice! Thanks and Regards, Aritra Basu
On Thu, Apr 23, 2026 at 9:17 PM Amogh Desai <[email protected]> wrote: > Hi Again, > > Created an issue tracking the migration of provider triggers from the old > CancelledError > pattern to the new on_kill() hook: > https://github.com/apache/airflow/issues/65733 > > Three triggers need updating — Dataproc, EMR Serverless, and Kubernetes. > Each is a > contained change in a single file. Good first contribution if you want to > get familiar with the > triggerer. > > Drop a comment on the issue if you want to pick one up. > > Thanks & Regards, > Amogh Desai > > > On Thu, Apr 23, 2026 at 7:40 PM Kaxil Naik <[email protected]> wrote: > > > Finally, this has been troubling users since Triggerer was added. > > > > On Thu, 23 Apr 2026 at 14:59, Elad Kalif <[email protected]> wrote: > > > > > This is great! well done! > > > > > > On Thu, Apr 23, 2026 at 4:42 PM Amogh Desai <[email protected]> > > wrote: > > > > > > > Hey All, > > > > > > > > Wanted to call out a new hook we added to BaseTrigger — `on_kill()`, > > PR: > > > > https://github.com/apache/airflow/pull/65590. > > > > > > > > *Why does it exist?* > > > > > > > > If your trigger polls an external job (BigQuery, Databricks, > Dataproc, > > > > etc.), there was no clean way to cancel that job when > > > > a user kills the deferred task, either by marking it as success, > > failure, > > > > or clearing it. `cleanup()` wasn't the right place — > > > > it fires on every trigger exit including triggerer restarts and > rolling > > > > deploys, so putting cancellation there would kill in-flight > > > > work on every redeploy. > > > > > > > > `on_kill()` solves this. It only fires when a user explicitly acts on > > the > > > > task. Not on restart, not on redistribution. > > > > > > > > *How it works* > > > > > > > > The triggerer passes a private sentinel via > `asyncio.Task.cancel(msg)` > > to > > > > distinguish user kills from other exits — no DB > > > > round-trip needed. There's a configurable timeout (*[triggerer] > > > > on_kill_timeout, default 30s*) so a slow external API call > > > > cannot block the triggerer. > > > > > > > > Databricks is updated as a reference: > > > > https://github.com/apache/airflow/pull/65672 > > > > > > > > Both `DatabricksExecutionTrigger` and > > > > `DatabricksSQLStatementExecutionTrigger` now implement it. > > > > > > > > *Who benefits from this?* > > > > > > > > If you are someone building or maintaining a trigger that manages > > > external > > > > work, you can consider adopting to this hook > > > > and implementing it for your trigger class, Airflow 3.3 will launch > > this. > > > > > > > > No compat guard is needed, Airflow < 3.2 the method just never gets > > > called. > > > > > > > > *Next steps* > > > > > > > > I pulled some data from the repo and these are candidates for this > > > > migration. > > > > > > > > Google Provider: DataprocSubmitTrigger and > > DataprocSubmitJobDirectTrigger > > > > in the google provider. Currently catches > > > > CancelledError in run() and calls hook.cancel_job(job_id) after > > checking > > > > safe_to_cancel(). > > > > > > > > Amazon Provider: EmrServerlessStartJobTrigger. Same pattern. > > > CancelledError > > > > handler with safe_to_cancel() calling > > > > hook.conn.cancel_job_run(). Straightforward migration. > > > > > > > > CNCF provider: KubernetesPodTrigger in cncf/kubernetes. Has an > > > > on_kill_action parameter controlling whether to > > > > delete/patch the pod. The cancellation logic lives in a > CancelledError > > > > handler. Move it to on_kill(). > > > > > > > > Happy to help with reviews for these. > > > > > > > > > > > > Thanks & Regards, > > > > Amogh Desai > > > > > > > > > >
