ColtenOuO opened a new pull request, #72272:
URL: https://github.com/apache/airflow/pull/72272
### Summary
`get_data_dependencies()` (the Assets dependency-graph API) shares a single
`processed_tasks` set between its producer-side and consumer-side BFS branches.
A task that both consumes one asset and produces another gets marked
"processed" by whichever branch the BFS reaches it from first, which silently
skips the batched lookup the *other* branch would have run — dropping any asset
only reachable through that skipped direction, with no error or warning that
the graph is incomplete.
### Example
Task `transform` reads asset `X` and writes assets `Y` and `Z`:
```mermaid
flowchart LR
X([asset X]) --> transform[[transform]]
transform --> Y([asset Y])
transform --> Z([asset Z])
```
Querying the dependency graph rooted at `Y`:
```mermaid
sequenceDiagram
participant Y as asset Y (query root)
participant BFS
participant X as asset X
Note over BFS: Round 1 — visiting Y
Y->>BFS: transform produces Y
BFS->>BFS: mark transform processed<br/>query its INLETS → finds X
Note over BFS: Round 2 — visiting X
X->>BFS: transform consumes X
BFS->>BFS: transform already processed<br/>→ OUTLET query skipped
Note over BFS: Z is never queried — silently missing from the graph
```
`transform` is discovered twice — once as a producer of `Y` (which queries
its *inlets*, finding `X`), and once as a consumer of `X` (which should query
its *outlets* to keep tracing downstream). Because both branches gate their
lookup on the same `processed_tasks` set, the second visit sees `transform`
already marked and skips the outlet query entirely — so `Z`, an outlet
unrelated to `X` or `Y`, never appears in the graph.
### Fix
Split the shared `processed_tasks` set into two independently-tracked sets,
`processed_inlet_tasks` and `processed_outlet_tasks`, so a task reached from
one direction still has the other direction's lookup run exactly once. The
final team-annotation step, which previously derived `all_dag_ids` from
`processed_tasks`, now derives it directly from the task nodes already
collected in the graph — the more direct source of truth, and unaffected by the
split.
### Testing
Added
`test_data_dependencies_finds_all_outlets_for_task_that_also_consumes_an_asset`,
reproducing the exact scenario above (task reads X, writes Y and Z; query
rooted at Y). Verified the test fails on the pre-fix code (`Z` missing from the
response) and passes after the fix.
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes — Claude Code (Sonnet 5) for writing test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
