abhijeets25012-tech opened a new pull request, #75263: URL: https://github.com/apache/airflow/pull/75263
When the scheduler checks task concurrency, it loads the Dag for each scheduled task instance. If a Dag cannot be deserialized, that exception can stop scheduling and cause the scheduler to crash again on restart. This change catches and logs non-database errors during that load, defers the affected task for the current selection pass, and allows the scheduler to continue processing other Dags. Database errors still propagate so the existing transaction retry behavior is preserved. Adds a regression test confirming that the affected task remains scheduled while a healthy Dag’s task is selected. closes: #75254 ### Testing - `airflow-core/tests/unit/jobs/test_scheduler_job.py::TestSchedulerJob::test_find_executable_task_instances_skips_dag_that_cannot_be_deserialized` - `airflow-core/tests/unit/jobs/test_scheduler_job.py::TestSchedulerJob::test_start_queued_dagruns_does_not_crash_on_dag_deserialization_error` - `prek run --files airflow-core/src/airflow/jobs/scheduler_job_runner.py airflow-core/tests/unit/jobs/test_scheduler_job.py` --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Codex (GPT-6) Generated-by: Codex (GPT-6) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
