rapsealk opened a new pull request, #65836: URL: https://github.com/apache/airflow/pull/65836
On HA scheduler deployments backed by MySQL, the scheduler's `SchedulerJobRunner.check_trigger_timeouts()` and the triggerer's `Trigger.clean_unused()` both issue bulk UPDATEs against `task_instance` rows but reach those rows via different indexes — `(state, trigger_timeout)` vs `(trigger_id)`. InnoDB therefore acquires row + gap locks in different orders for the two queries, producing classic A→B / B→A deadlocks once a deferrable workload has overlapping rows in flight (especially with multiple scheduler replicas). This change reshapes both writers to: 1. `SELECT id ... ORDER BY id LIMIT N FOR UPDATE SKIP LOCKED` 2. `UPDATE task_instance ... WHERE id IN (...)` so that both paths take row locks in deterministic primary-key order, eliminating the cross-index deadlock. `SKIP LOCKED` additionally keeps concurrent scheduler replicas from blocking on each other. Behaviour is unchanged: same predicates, same SET clauses, just batched and locked. Two regression tests cover convergence when more rows match than fit in a single batch (the batch size is monkeypatched to 2 in the tests). closes: #65818 --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Claude Opus 4.7 (1M context) Generated-by: Claude Opus 4.7 (1M context) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
