GitHub user hamedrabah added a comment to the discussion: [Discussion] DAG-scoped import errors for factory/driver files (e.g. dag-factory)
Airflow's parser unit is the Python file. An uncaught exception means that module did not finish importing, so Airflow cannot reliably treat some DAG objects from that module as healthy while also recording a native import failure for the file. The stale-DAG behavior follows from that boundary. Today, a factory that must preserve healthy DAGs has two practical choices: 1. Catch errors per configuration, register the valid DAGs, and emit explicit diagnostics for invalid configurations through logs/metrics or a small diagnostic DAG. 2. Generate separate driver files, one per configuration or bounded group. Then Airflow's existing file-level import error maps cleanly to the affected DAGs and leaves the other files healthy. The second option gives the strongest isolation. Catching exceptions in one large driver keeps 497 DAGs alive, but the failures cannot also appear as native import errors without a new API. I would not overload the existing file import-error field with partial-success semantics because schedulers currently use that state to decide whether every DAG from the file is stale. A new structured “bundle/factory diagnostics” channel could support partial success without changing what an import error means. GitHub link: https://github.com/apache/airflow/discussions/70119#discussioncomment-18148546 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
