The GitHub Actions job "Check newsfragment PR number" on airflow.git/fix/scheduler/dbdagbag-unbounded-cache has succeeded. Run started by GitHub user jason810496 (triggered by jason810496).
Head commit for run: 4b308b23b5bc96fbdee46db6c24b4e9271a47094 / LIU ZHE YOU <[email protected]> Bound the scheduler's deserialized Dag cache The scheduler kept every Dag version it deserialized in a mapping that never evicted, so a long-running scheduler grew with the number of versions it had ever seen until it was restarted or OOM killed. Deployments that redeploy Dags frequently accumulate versions fastest and hit this soonest. A least-recently-used cap is the only thing that bounds this outright. An idle timeout would not: the scheduler re-checks an entry on each lookup, which re-arms its expiry, so a timeout reclaims a version only once its runs finish and it stops being requested, leaving memory a function of the concurrently active set rather than a fixed ceiling. Deliberately not configurable here, so the fix stays small enough to cherry-pick. Cache activity currently reports under the existing api_server.dag_bag.* metrics; a scheduler-specific namespace, along with configuration, follows separately. closes: #69001 Report URL: https://github.com/apache/airflow/actions/runs/32218840844 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
