meel-ayush opened a new pull request, #76559:
URL: https://github.com/apache/airflow/pull/76559
### Description
Closes: #76553
When bundle initialization, refresh, or file discovery fails in
`DagFileProcessorManager` (such as a git connection timeout, repository
authentication failure, rate limiting, or disk permission issues), the
exception was caught and logged to daemon logs, but no record was persisted to
the metadata database.
Because `ParseImportError` received 0 records, the Airflow Web UI reported 0
import errors and displayed the environment as fully healthy, even though Dag
synchronization from the bundle had halted completely.
This PR ensures bundle-level refresh and discovery failures are persisted to
the metadata database:
- Records and updates `ParseImportError` for the affected bundle when
`bundle.initialize()`, `bundle.refresh()`, or `_find_files_in_bundle()` raises
an unhandled exception.
- Persists the failure timestamp and full stacktrace, making bundle sync
errors visible on the UI dashboard and in the DAG Bundles import error column.
- Automatically purges the bundle-level `ParseImportError` once a subsequent
bundle refresh succeeds.
### Tests
- Unit tests added in
`airflow-core/tests/unit/dag_processing/test_manager.py`:
-
`test_refresh_dag_bundles_refresh_failure_records_and_clears_import_error`:
verifies that refresh exceptions record a `ParseImportError` and that
subsequent successful refreshes clear the error.
- `test_refresh_dag_bundles_discovery_failure_records_import_error`:
verifies that file discovery exceptions record a `ParseImportError`.
### AI Usage
- [ ] Yes (please specify the tool below)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
