steveahnahn opened a new pull request, #72668:
URL: https://github.com/apache/airflow/pull/72668
The `all_success` trigger rule can silently mark mapped task instances
REMOVED although their own expansion input never changed. The instances never
execute and the run still completes green.
Reproduced through the real scheduling-decision path. `notify` maps over an
unchanged 3-element literal downstream of a mapped task that shrank from 5 to 2
on a rerun:
```
GATE notify before decision: {0: None, 1: None, 2: None}
GATE notify after decision: {2: 'removed', ...}
```
### Root cause
Since 2.4 the flag-upstream-failed path infers staleness by comparing an
instance's `map_index` against the aggregate upstream success count. That proxy
only equals the instance's own new expansion length when the task expands 1:1
over exactly the counted upstream and every upstream instance is done. For any
other topology the comparison removes valid instances: a static list downstream
of a shrunk mapped task, a zip, several upstream tasks summed into one count,
or an evaluation that runs while upstream instances are still finishing. The
dep's own pass arithmetic agrees the rule is satisfied in these cases
(`num_failures` is 0).
### Fix
The inference now fires only for a strict 1:1 mapping, where the task's sole
expansion reference is exactly the counted upstream and nothing multiplies the
length, and only once the upstream is fully done, which also closes the race.
The setup-constraint path no longer infers removal at all: a task can only
expand over a direct upstream, and that path counts indirect setups only. The
rerun-deadlock scenario the inference was introduced for still resolves; its
tests pass unchanged.
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes — Claude Code (Fable 5)
Generated-by: Claude Code (Fable 5) following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
