bingqin2 commented on PR #72906: URL: https://github.com/apache/airflow/pull/72906#issuecomment-5651305546
A quick measurement of the repeated call, since that is what #72513 is about. Setup: the `configure_testing_dag_bundle` fixture with the parent/child Dags from `test_dag_test_with_trigger.py` plus N generated one-task sibling Dags; timing `parent.test()` after the first call (the first call still walks the bundle on both sides, 4.2 s vs 4.6 s, within noise). | siblings in the bundle | main (6a6a24f2) | this PR | |---|---|---| | 50 (the unit-test `dags` folder) | 0.35–0.48 s | 0.08–0.10 s | | 300 | 0.58–0.81 s | 0.10–0.11 s | On main every later call re-imports each sibling file and runs it through the sync again (the serialized hash is unchanged, so the DB write itself is skipped, which is why one-task siblings stay cheap). On this branch the call parses one file, and the cost no longer grows with the bundle. Real Dag files with heavier imports or many tasks widen the gap. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
