kaxil commented on code in PR #72909:
URL: https://github.com/apache/airflow/pull/72909#discussion_r4007954104
##########
airflow-core/docs/administration-and-deployment/dag-bundles.rst:
##########
@@ -226,6 +226,39 @@ Starting Airflow 3.0.2 git is pre installed in the base
image. However, if you a
ENV GIT_PYTHON_REFRESH=quiet
+Monitoring Dag bundles
+----------------------
+
+The **Dag Bundles** page under *Browse* shows, for each bundle, the version
Airflow currently holds,
+when a Dag processor last refreshed it, its import-error count, and whether it
is still configured
+-- so you can tell whether a commit has been picked up without reading the Dag
source. The page
+refreshes itself while open, and the same data is available at ``GET
/api/v2/dagBundles`` for a
+deployment pipeline to poll.
+
+``version`` is whatever the bundle reports; for a Git Dag bundle it is the
commit SHA of the tracking
+ref. Bundles that do not support versioning, such as ``LocalDagBundle``,
report ``null``.
+
+``last_refreshed`` is the last *successful* refresh and advances whether or
not the version changed.
+The cadence is the bundle's :ref:`config:dag_processor__refresh_interval`,
checked every
+:ref:`config:dag_processor__bundle_refresh_check_interval`.
+
+Some caveats:
+
+* A failing refresh looks like one that is not due, since a refresh that
raises is logged and leaves
+ ``last_refreshed`` at its last success. Check the Dag processor logs, and
+ ``/api/v2/jobs?job_type=DagProcessorJob`` for a live processor.
+* A new version does not mean the new Dags are running, because bundles are
refreshed before their
+ Dags are parsed -- read the import-error count alongside it.
Review Comment:
You are right, and the timeout leg is the one I had missed.
`_kill_timed_out_processors` sets `import_errors=1` on the in-memory
`DagFileStat` and never writes a `ParseImportError` row, so a file that hangs
past `dag_file_processor_timeout` reads as `import_error_count: 0` while
parsing is genuinely broken.
Reworded in 5c3248b. The section now says outright that the version and the
import-error count together cannot confirm a particular commit parsed and
should not be read as a deployment-health check, and names all three reasons:
errors are not recorded against a version, a timed-out parse writes no row at
all, and the count only covers errors the reader is allowed to see. It adds
that zero therefore means "nothing I can show you" rather than "this commit is
healthy", and points at `/api/v2/dags/{dag_id}/dagVersions` for actually tying
a running Dag to a commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
