vatsrahul1001 opened a new pull request, #75745:
URL: https://github.com/apache/airflow/pull/75745
Backport of the scheduler-crash fix from #75263 to `v3-4-test`, **adapted**
to the branch.
### Problem (present in 3.4.0)
In `SchedulerJobRunner._task_concurrency_allows_execution`, the call to
`get_dag_for_run()` is unguarded. If a single Dag cannot be deserialized (e.g.
a Variable referenced at parse time is missing, or a serialized-Dag
incompatibility), the exception propagates and **crashes the whole scheduler
loop** — so no Dag gets scheduled, not just the broken one. `v3-4-test` has
this unguarded call today.
### Fix
Wrap the load:
```python
try:
serialized_dag = self.scheduler_dag_bag.get_dag_for_run(...)
except DBAPIError:
raise # DB errors must propagate for
transaction-retry handling
except Exception:
self.log.exception("Failed to load Dag %s for run %s, skipping it",
dag_id, run_id)
return False # skip this task instance, keep scheduling
other Dags
```
### Why this is adapted, not a straight cherry-pick
The original #75263 marks the failed Dag run in a `starved_dag_runs` set so
sibling tasks of the broken run are skipped without re-loading/re-logging. That
set was introduced by **#70307** ("Fill scheduler task batches after
concurrency rejection"), which is milestoned **3.4.1** and is **not** on
`v3-4-test`. Cherry-picking #75263 as-is fails to compile (`starved_dag_runs`
undefined) and its tests reference `MAX_TI_REFILL_QUERIES_PER_LOOP`, also from
#70307.
This backport keeps the crash-prevention (the part that matters for the
release) and **drops the `starved_dag_runs` optimization**. Functionally
identical for correctness — the only difference is that without the set,
sibling tasks of a broken run are re-examined (and re-logged) in the same loop
instead of being short-circuited. The run-starving optimization will arrive
naturally if/when #70307 + #75263 land together in 3.4.1.
### Tests
Two of #75263's three tests apply unchanged and are included:
-
`test_find_executable_task_instances_skips_dag_that_cannot_be_deserialized` — a
Dag that fails to deserialize is skipped and logged; unrelated Dags still get
scheduled.
- `test_find_executable_task_instances_reraises_db_errors_from_dag_load` — a
`DBAPIError` during load propagates (so transaction-retry handling still runs).
The third upstream test exercises the dropped `starved_dag_runs`
run-starving optimization and is intentionally omitted.
Local validation: the `reraises` test passes under `uv run`; the `skips`
test uses `caplog` and is blocked only by a local shared-namespace wiring
issue, so it is validated here in CI (it is the same caplog pattern that passed
in #75263's own CI).
---
##### Was generative AI tooling used to co-author this PR?
- [x] Yes - Claude Code (Opus 4.8)
Generated-by: Claude Code (Opus 4.8) following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
