anzinmhd opened a new pull request, #74188:
URL: https://github.com/apache/airflow/pull/74188
<!-- SPDX-License-Identifier: Apache-2.0
https://www.apache.org/licenses/LICENSE-2.0 -->
## What is the issue?
When a `ShortCircuitOperator` is used inside a mapped task group, transitive
downstream tasks can incorrectly execute even when the short-circuit condition
is `False`.
The `ShortCircuitOperator` records all downstream tasks that should be
skipped in its `SkipMixin` XCom. However, `NotPreviouslySkippedDep` only checks
the direct upstream relatives of a task.
As a result, a transitive downstream task with a permissive trigger rule
such as `all_done` can run even though it was explicitly recorded as skipped by
the upstream `ShortCircuitOperator`.
For example:
```text
group.gate
|
group.a
|
group.b (all_done)
```
For a short-circuited map index, `group.a` is skipped correctly, but
`group.b` can incorrectly run.
## What does this PR change?
Update `NotPreviouslySkippedDep` to inspect all upstream relatives instead
of only direct upstream relatives.
This allows the dependency to find the upstream `SkipMixin` task and honor
its recorded skip decision for transitive downstream tasks.
## Tests
Added a regression test covering:
* A `ShortCircuitOperator` inside a mapped task group.
* A transitive downstream task using `TriggerRule.ALL_DONE`.
* Skip decisions for a specific map index.
* Ensuring the affected map index is skipped.
* Ensuring the unaffected map index is not skipped.
## Validation
The following checks were run successfully:
* Focused `NotPreviouslySkippedDep` unit tests.
* Ruff checks for the modified files.
* Manual reproduction of the reported issue.
The manual reproduction now results in:
```text
Map 0:
gate -> SUCCESS
a -> SUCCESS
b -> SUCCESS
Map 1:
gate -> SUCCESS
a -> SKIPPED
b -> SKIPPED
```
closes: #74175
---
##### Was generative AI tooling used to co-author this PR?
* [x] Yes (ChatGPT)
<!--
Generated-by: ChatGPT following the guidelines:
https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions
-->
<!--
Generated-by: [Tool Name] following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
-->
---
* Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
for more information. Note: commit author/co-author name and email in commits
become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment:
`{pr_number}.significant.rst`, in
[airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
You can add this file in a follow-up commit after the PR is created so you
know the PR number.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
