1fanwang opened a new pull request, #72927:
URL: https://github.com/apache/airflow/pull/72927
After serialized Dag metadata is lost, operators need to restore missing
Dags without waiting for every healthy file to parse again. This adds
`--only-missing` to `airflow dags reserialize`, with the existing
`--bundle-name` option for scope.
Recovery reads recorded paths within each bundle and commits each file
independently. Existing serialized versions remain available, and Dag
processors still handle removals. Stale or unregistered Dags and missing
historical versions are outside this mode's scope.
The writer also restores a missing serialized row when its unused Dag
version survives. The current command otherwise returns successfully without
restoring that Dag.
## Testing
Real Dag files and native CLI subprocesses against SQLite. The fixture
serializes the Dags normally before removing two serialized rows.
| Case | Files parsed | Serialized Dags afterward |
|---|---:|---:|
| Upstream main, full reserialize | 4 | 4 of 6 |
| This PR, full reserialize | 4 | 6 of 6 |
| This PR, only missing | 1 | 6 of 6 |
A generated 70-Dag sample also preserves existing row IDs, hashes, and
version references while reducing the files parsed from 68 to 1.
<details><summary>Commands and raw logs</summary>
From a configured Airflow test environment:
```bash
export AIRFLOW_HOME="$(mktemp -d)"
export AIRFLOW_CONFIG="$AIRFLOW_HOME/airflow.cfg"
export
AIRFLOW__DATABASE__SQL_ALCHEMY_CONN="sqlite:///$AIRFLOW_HOME/airflow.db"
.venv/bin/python -m pytest
airflow-core/tests/unit/cli/commands/test_dag_reserialize.py -k native_cli
--with-db-init -q -s --tb=short
```
Before, on main:
```text
{"mode": "full", "parsed_files": 4, "serialized_dags": 4}
```
With this PR:
```text
{"mode": "full", "parsed_files": 68, "serialized_dags": 70,
"retained_rows_unchanged": true}
{"mode": "only-missing", "parsed_files": 1, "serialized_dags": 70,
"retained_rows_unchanged": true}
```
</details>
Coverage includes bundle scoping, relocated bundle paths, ZIP and
single-file bundles, partial recovery, compressed metadata, and deadline
reconstruction.
---
##### Was generative AI tooling used to co-author this PR?
- [x] Yes: GitHub Copilot CLI (GPT-6 Astra).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
