GitHub user cookesan added a comment to the discussion: S3 Dag Bundle won't remove files removed on remote S3
The current source says stale local files should be deleted. `S3DagBundle.refresh()` calls `S3Hook.sync_to_local_dir(..., delete_stale=True)`, and `sync_to_local_dir()` has a stale-file cleanup path that removes local files not present in the current S3 object list. So I would first check the exact Airflow and `apache-airflow-providers-amazon` versions in the environment. If your installed provider version predates that `delete_stale=True` behavior, upgrading the Amazon provider may be the fix. If you are already on a version with that code, then this is probably worth a small reproducible bug report: bucket prefix, bundle config, provider version, whether the deleted file is under the configured prefix, and debug logs from `S3Hook.sync_to_local_dir()`. GitHub link: https://github.com/apache/airflow/discussions/64775#discussioncomment-16657270 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
