GitHub user cookesan added a comment to the discussion: [Dag Processor] Parsing is skipped when there are too many callbacks
You found the relevant branch. In the current processor code, a parse request with callback requests builds the `DagBag`, executes the callbacks, and then returns `None` instead of serializing DAGs. I would be cautious about changing only the `return None`. It may make your environment fresher, but it also changes the meaning of a callback request from "load the DAG file to execute callbacks" into "also publish a parse result", which could affect processor load and stale-DAG behavior. The underlying issue sounds more like callback work competing with normal parse freshness. A cleaner PR would probably separate or prioritize those paths: for example, avoid letting a callback backlog consume parse slots needed for DAG updates, or make callback execution budgeted/observable. The useful evidence for a PR would be processor counts, callback queue size, parse interval, how many callback requests arrive per minute, and a minimal reproduction showing a normal DAG edit delayed behind callback-heavy files. As an operational workaround, I would reduce callback volume or increase processor capacity before carrying a production patch that always serializes after callbacks. GitHub link: https://github.com/apache/airflow/discussions/64580#discussioncomment-16657273 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
