This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a commit to branch v3-0-test in repository https://gitbox.apache.org/repos/asf/airflow.git
commit e06fb664c044f377d2e8f8d6d5da6f5a4a40215d Author: rtrindvg <[email protected]> AuthorDate: Mon Jun 2 05:07:59 2025 -0300 Add "disappearing DAGs" section on FAQ doc (#49987) --------- Co-authored-by: Amogh Desai <[email protected]> (cherry picked from commit b89044da97efba3ef43ec6c2441c7026ab61fc69) --- airflow-core/docs/faq.rst | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/airflow-core/docs/faq.rst b/airflow-core/docs/faq.rst index 8727deac08e..9eb5fa79ad0 100644 --- a/airflow-core/docs/faq.rst +++ b/airflow-core/docs/faq.rst @@ -206,6 +206,37 @@ until ``min_file_process_interval`` is reached since DAG Parser will look for mo return dag +What to do if you see disappearing DAGs on UI? +---------------------------------------------- +There are several reasons why DAGs might disappear from the UI. Common causes include: + +* **Total parsing of all DAGs is too long** - If parsing takes longer than :ref:`config:core__dagbag_import_timeout`, + files may not be processed completely. This often occurs when DAGs don't follow + :ref:`DAG writing best practices<best_practice:writing_a_dag>` like: + + * Excessive top-level code execution + * External system calls during parsing + * Complex dynamic DAG generation + +* **Inconsistent dynamic DAG generation** - DAGs created through + :doc:`dynamic generation </howto/dynamic-dag-generation>` must produce stable DAG IDs across parses. + Verify consistency by running ``python your_dag_file.py`` repeatedly. + +* **File processing configuration issues** - A certain combination of parameters may lead to scenarios which certain DAGs are less likely to be processed at each loop. Check these parameters: + + * :ref:`config:dag_processor__file_parsing_sort_mode` - Ensure sorting method matches your sync strategy + * :ref:`config:dag_processor__parsing_processes` - Number of parallel parsers + * :ref:`config:scheduler__parsing_cleanup_interval` - Controls stale DAG cleanup frequency + * :ref:`config:scheduler__dag_stale_not_seen_duration` - Time threshold for marking DAGs as stale + +* **File synchronization problems** - Common with git-sync setups: + + * Symbolic link swapping delays + * Permission changes during sync + * `mtime` preservation issues + +* **Time synchronization issues** - Ensure all nodes (database, schedulers, workers) use NTP with <1s clock drift. + DAG construction ^^^^^^^^^^^^^^^^
