The GitHub Actions job "Tests (AMD)" on 
airflow.git/delete-dag-drop-unused-session-sync has succeeded.
Run started by GitHub user ColtenOuO (triggered by ColtenOuO).

Head commit for run:
79f9de732db022254f6a17582eaef074a0290797 / ColtenOuO <[email protected]>
Reduce memory used when deleting a Dag with a large history

delete_dag forced SQLAlchemy's "fetch" synchronization strategy on every bulk
delete it issues. That strategy reads the primary key of every deleted row back
from the database so it can mark matching in-memory objects as deleted, but the
session holds nothing beyond the Dag's own DagModel row — the keys were matched
against an effectively empty identity map and discarded, once per table with a
dag_id column.

The cost scaled with the Dag's history rather than with the number of objects
actually needing synchronization: roughly 211 bytes of transient Python heap per
deleted row on PostgreSQL, or about 1 GiB in the API server for a Dag with five
million task instances.

The default strategy evaluates the criteria in Python against the objects 
already
loaded, so synchronization still happens without a round-trip sized by the row
count.

Report URL: https://github.com/apache/airflow/actions/runs/31016904494

With regards,
GitHub Actions via GitBox


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to