shivaam opened a new pull request, #69073: URL: https://github.com/apache/airflow/pull/69073
Split out from #68961. `Deadline.handle_miss()` adds deadline context and executor routing fields to `callback.data`, then persists the callback row. The current code mutates nested JSON in place, so SQLAlchemy may not mark the `ExtendedJSON` payload dirty. That leaves reloaded callback rows without the data the scheduler expects, such as `dag_run_id` for executor callbacks. This updates `handle_miss()` to reassign the callback data payload when adding context/routing fields. It also stores the deadline id in callback context as a string so triggerer callback kwargs stay serialization-safe. Tests added: - commit/reload coverage for triggerer callback context persistence - commit/reload coverage for executor callback routing data persistence Validation: - `uv run --project airflow-core pytest -q airflow-core/tests/unit/models/test_deadline.py --with-db-init` -> `55 passed, 1 warning` - `uv run --project airflow-core pytest -q airflow-core/tests/unit/models/test_deadline.py::TestDeadline::test_handle_miss airflow-core/tests/unit/models/test_deadline.py::TestDeadline::test_handle_miss_persists_triggerer_callback_context airflow-core/tests/unit/models/test_deadline.py::TestDeadline::test_handle_miss_persists_executor_callback_routing_data --with-db-init` -> `3 passed, 1 warning` - `uv run ruff check airflow-core/src/airflow/models/deadline.py airflow-core/tests/unit/models/test_deadline.py` -> `All checks passed!` - `uv run ruff format --check airflow-core/src/airflow/models/deadline.py airflow-core/tests/unit/models/test_deadline.py` -> `2 files already formatted` - `git diff --check` -> passed I also checked this with a local real Airflow/Postgres daemon harness. On upstream `main`, the sync/executor deadline callback row was marked queued but the persisted `callback.data` lacked the routing/context fields. With this patch, the sync callback row persisted `dag_run_id`, `deadline_id`, and context, and the callback marker was written. The async/triggerer path succeeded in that Postgres daemon check on mainline; the async test here is defensive persistence/serialization coverage, not a claim that async callbacks always fail live. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
