hiSandog commented on issue #69001: URL: https://github.com/apache/airflow/issues/69001#issuecomment-4830536164
The memray trace pointing at `_get_dag` growing 465→670MB lines up with the scheduler's `DBDagBag` having no bound — unlike the API server, which got a `dag_cache_size`/`dag_cache_ttl` knob in #60804. Mirroring that same config into `[scheduler]` (and respecting it in the `SchedulerJobRunner` `DBDagBag(load_op_links=False)` path) would cap the growth directly. Worth noting `num_runs=-1` makes the scheduler run indefinitely, so without a TTL the cache only ever grows across the process lifetime. The API-server fix is a good template here since the access pattern is similar. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
