Vamsi-klu opened a new pull request, #72134: URL: https://github.com/apache/airflow/pull/72134
related: #65818 related: #71391 related: #72062 related: #70961 Not a second retry. #71391 covers `@retry_db_transaction` on `submit_event`/`submit_failure`. This PR is the lock-order / ownership split. The issue body swaps the writers. On current main: - scheduler `check_trigger_timeouts`: unbounded `UPDATE ... state=deferred AND trigger_timeout < now()` - triggerer `Trigger.clean_unused`: unbounded `UPDATE ... state!=deferred AND trigger_id IS NOT NULL` - triggerer `submit_event`: single-row PK UPDATE Predicates are logically disjoint. InnoDB still deadlocks because one scan uses `ti_state` and the other uses `ti_trigger_id`. This PR: - While `state=deferred`, only the owning triggerer writes `trigger_id` / `next_*`. Scheduler timeout becomes a bounded `ORDER BY id LIMIT 100 FOR UPDATE SKIP LOCKED` fallback for unassigned or dead TriggererJob heartbeats. Healthy assigned + past `trigger_timeout` is left alone (the triggerer already maps cancel-past-timeout to `submit_failure`). - `clean_unused` no longer UPDATEs `task_instance`. MySQL DELETE re-checks `~assets / ~callback / ~task_instance` so a deferral between SELECT and DELETE cannot CASCADE-delete the TI (#72062). - DEFERRED-exit paths NULL `trigger_id` in the same transaction: `clear_task_instances`, `TaskInstance.set_state`, DagRun timeout SKIPPED, mapped REMOVED. Execution API skip-downstream is left for a follow-up (datamodel regen hook). - `submit_event` / `submit_failure` SELECT `FOR UPDATE SKIP LOCKED` and no-op if another writer took the row. No new retry loop. Timeout SET is unchanged: `SCHEDULED`, `__fail__`, `TRIGGER_TIMEOUT`, `scheduled_dttm`, `trigger_id=None`. Tests that fail on revert: - healthy assigned triggerer + past timeout is not flipped - dead triggerer fallback still times out - `clean_unused` issues no `UPDATE task_instance` - leftover SUCCESS+trigger_id is no longer bulk-unlinked - MySQL two-step DELETE keeps a trigger that gained a TI after the candidate SELECT - `clear_task_instances` NULLs `trigger_id` - existing retry-on-OperationalError timeout test still passes I will stay on the review thread. The last two lock-order PRs (#65836, #65920) died of staleness. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes Generated-by: Cursor Grok 4.6 following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
