The GitHub Actions job "Tests (ARM)" on airflow.git/main has succeeded.
Run started by GitHub user potiuk (triggered by potiuk).

Head commit for run:
cad6220034e87c586eb87e14b5a75f4b82984fd0 / Daniel Standish 
<[email protected]>
Speed up dynamic task mapping expansion (#69565)

Expanding a mapped task creates the N downstream task instances inside
`DagRun.update_state`. Two `N+1` query patterns there turned a wide fan-out
into a scheduler stall that scaled linearly with map width: a per-index
`session.merge()` to persist each new instance, and a per-index `dag_run`
`SELECT` when dependency evaluation later called `get_dagrun()` on instances
whose dag_run relationship was never loaded.

Persist the instances with a batched `session.add()`/`flush()` and prime the
shared `dag_run` relationship once during expansion, so both scale with a
constant number of queries instead of with map width.

We could have also defensively added a `set_committed_value` on the dagrun attr 
for the index-0 case but it's not necessary as is because the dag run object is 
cached for that reused TI object.

Report URL: https://github.com/apache/airflow/actions/runs/29363616049

With regards,
GitHub Actions via GitBox


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to