o-nikolas commented on PR #46621: URL: https://github.com/apache/airflow/pull/46621#issuecomment-3090778221
@jedcunningham > My 2c, giving bundles somewhere to store this extra stuff makes sense. :heart: > I'd rather we let the bundles own id creation though - if there is one that is meaningful already, we should use that. I think a consistent version id format is better than a wild west of IDs from each provider, but I'm happy to disagree and commit on this one. I see where you're coming from on the id sometimes being useful (but I really only see git as the example of that). > This is what we have today actually, it's just very limited in size, particularly when you consider the "I need to store the version of every object in a bucket" type situation. So, providing somewhere to store it makes sense, but we don't have a natural place for it today (not hard to add though). Yeah, today the version field is both a unique ID and the data actually needed to recreate the bundle version represented by that id (Git, again, is the hallmark example of this). But the S3 case we'd want a unique ID that is more of a UUID Primary Key which is then used to fetch the manifest needed to recreate the bundle version on the filesystem. Since the manifest will be large(ish) and we don't want it displayed in the UI or any other user accessible location. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
