Team,


We just created 'AIP-36 DAG Versioning' on Confluence and would very much
appreciate feedback and suggestions from the community.



https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-36+DAG+Versioning



The DAG Versioning concept has been discussed on multiple occasions in the
past and has been a topic highlighted as part of Airflow 2.0 as well. We at
Astronomer have heard data engineers at several enterprises ask about this
feature as well, for easier debugging when changes are made to DAGs as a
result of evolving business needs.


As described in the AIP, we have a proposal focused on ensuring that the
visibility behaviour of Airflow is correct, without changing the execution
behaviour. We considered changing the execution behaviour as well, but
decided that the risks in changing execution behavior were too high as
compared to the benefits and therefore decided to limit the scope to only
making sure that the visibility was correct.


We would like to attempt this based on our experience running Airflow as a
service. We believe that this benefits Airflow as a project and the
development experience of data engineers using Airflow across the world.


 Any feedback, suggestions, and comments would be greatly appreciated.



Best Regards,


Kaxil Naik, Ryan Hamilton, Ash Berlin-Taylor, and Vikram Koka

Reply via email to