hkc-8010 opened a new issue, #66300: URL: https://github.com/apache/airflow/issues/66300
### Apache Airflow version 3.2.0 ### What happened? We hit a case where a DAG was being actively parsed from the current bundle path, but the serialized DAG metadata still pointed to an older bundle path. In our case, DAG-level behavior was failing before a manual serialized-DAG refresh, then started working again after we deleted the serialized row and let Airflow recreate it. There is still some uncertainty about the exact internal callback execution path, but the stale serialized metadata was definitely present, and refreshing it correlated directly with the fix. We also saw repeated API/UI-side errors for the same DAG version while this was happening: ### What you think should happen instead? If a DAG's changes, the serialized DAG row should be updated to reflect the new path even when the serialized DAG content hash is unchanged. At minimum, should not stay stale indefinitely when the DAG is being actively reparsed from a new location. ### How to reproduce We do not yet have a minimal standalone repro, but this is the observed pattern: 1. Run Airflow with versioned or changing DAG bundle paths. 2. Keep the DAG definition effectively unchanged so the serialized DAG hash remains the same. 3. Move the same DAG to a new bundle path. 4. Observe that and current parsing reflect the new path, but may still point to the old path. 5. DAG-level callback behavior and some UI/version-history lookups may then behave incorrectly. ### Operating System Linux ### Versions of Apache Airflow Providers Not central to the issue ### Deployment Custom / Astro Hosted runtime based on Airflow 3.2.0 ### Deployment details Observed on Airflow 3.2.0 with versioned DAG bundle paths. ### Anything else? A few concrete observations from the incident: - was current and pointed at the active bundle path. - was current. - was still old. - still pointed at the old bundle path. - The old bundle path no longer existed on the current pod. - DAG-level failure callbacks did not appear to fire for failed runs before refresh. - After deleting the serialized DAG row and letting Airflow recreate it, the serialized updated to the current path and the next customer test worked. The serialization logic appears relevant here. In , the row is skipped when and are unchanged: But is set separately from when building the serialized row: So if only changes while the serialized DAG content hash stays the same, the row may never be rewritten. ### Are you willing to submit PR? Yes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
