jason810496 commented on PR #46621: URL: https://github.com/apache/airflow/pull/46621#issuecomment-3072211401
Hi all, Would it be a good idea to introduce a `RemoteDagBundle` interface (or perhaps extend `BaseDagBundle` or add a common utility in the `bundles.base` module) to support **versioning** for remote Dag bundles like `S3DagBundle`? We could support bundle versioning using one of the following approaches (potentially controlled by an `enable_versioning` parameter in `RemoteDagBundle`): 1. **Content-based hash**: Calculate a hash of the bundle by hashing each Dags file and incorporating the directory structure, then return a final hash value as the version. 2. **Timestamp-based version**: Use the last modified timestamp of the bundle. Most cloud providers support a `LastModified` or `updated` metadata field that we can leverage. If this sounds reasonable, I will open a new issue to track this feature and raise PR for it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
