Re: [DISCUSS] AIP-63, AIP-64, and AIP-65: DAG Versioning

In context of AIP-71 - slightly directing your attetion there for discussion purposes I think it would be nice to do a dag = dag_load(ObjectStoragePath("dagfs://mydag?version=1)) Having dag versioning as an fs implementation would open up additional interesting avenues for DAG manipulation. BT

Re: [DISCUSS] AIP-63, AIP-64, and AIP-65: DAG Versioning

Agreed. When Jed and team wrote the AIP, we intentionally limited the scope to DAGs since the AIPs were already really large, but the intention is to extend the concept to datasets. Funny that you bring up point #2. A few of us met last week to talk about DAG Versioning, and that use-case came up

Re: [DISCUSS] AIP-63, AIP-64, and AIP-65: DAG Versioning

Yes to both the below questions @Elad Kalif . The upcoming Data-Awareness AIPs the first one and the 2nd should be covered by AIP-66 once it is out of draft. 1. Should datasets be also versioned? > 2. Should we support executing more than 1 DAG version at a given time? On Tue, 28 May 2024 at 10:

Re: [DISCUSS] AIP-63, AIP-64, and AIP-65: DAG Versioning

I have a general question about (maybe somehow related to the DAG Bundle concept introduced in the AIPs) The way I see it DAGs are tightly coupled with Datasets. Tasks take dependency on dataset or/and produce a dataset. We are focused on the versions of the code (DAG) but to make this play nicely

[DISCUSS] AIP-63, AIP-64, and AIP-65: DAG Versioning

Hello everyone! I'm excited to start a discussion around DAG Versioning in Airflow. It's been the most requested feature in the last 3 community surveys! AIP-63: DAG Versioning As this topic quickly becomes rather lar