idan192 opened a new issue, #69841: URL: https://github.com/apache/airflow/issues/69841
### Apache Airflow version 3.3.0 and current `main` ### What happens The triggerer supervisor loads the pinned serialized Dag for every task-associated trigger so it can determine whether the task uses `start_from_trigger`. If that serialized Dag exists but no longer contains the deferred task, `get_task()` raises `TaskNotFound` from workload construction. The exception escapes the per-batch loop, so one stale task instance prevents unrelated healthy triggers in the same batch from starting and can repeatedly restart the triggerer supervisor. ### Sanitized reproducer 1. Create two deferred task instances with ordinary triggers. 2. Leave one task instance linked to a Dag version whose serialized Dag no longer contains that task ID. 3. Leave the second task instance and its pinned Dag version valid. 4. Let the triggerer claim both trigger rows in one polling cycle. The first call to `LazyDeserializedDAG.get_task()` raises `TaskNotFound`. The healthy trigger is not constructed during that supervisor pass. ### Expected behavior A stale task instance must not abort construction of unrelated trigger workloads. A backportable fix can treat this narrow lookup failure as an ordinary context-free deferral, matching behavior before triggerer-side `start_from_trigger` context loading was introduced. The durable fix should persist whether each trigger requires Dag context. Ordinary deferrals can then avoid serialized-Dag lookup entirely, direct-to-triggerer tasks can require their pinned Dag data, and legacy rows can self-classify once. If a context-required row points to a serialized Dag that lacks its task, the task instance should become `REMOVED` and unlink from the trigger without blocking healthy workloads. ### Acceptance criteria - One stale task-associated trigger cannot abort the triggerer supervisor batch. - Ordinary deferrals do not query or deserialize their Dag. - Context-required triggers never run with missing or unrendered Dag context. - Legacy rows remain safe during rolling upgrades. --- Drafted-by: Codex (GPT-5) (no human review before posting) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
