Adam Trump created AIRFLOW-5570: ----------------------------------- Summary: Allow to store DAGs on GIT Key: AIRFLOW-5570 URL: https://issues.apache.org/jira/browse/AIRFLOW-5570 Project: Apache Airflow Issue Type: New Feature Components: core, DAG, scheduler Affects Versions: 1.10.5 Reporter: Adam Trump
Today it is possible to store the DAGs only on a specific folder, which is configurable. I'd like to suggest to allow storing the DAGs on GIT. This feature comes from the fact that Airflow already have this functionallity - under the KubernetesExecutor. Thus, it shouldn't take much time moving it out of that executor, to the Airflow scheduler. When you run Airflow as a cluster on different machines, since all of the components (webserver, scheduler and worker) needs access to the DAGs file, there is always a need of accessable mount. It can be very helpful if we could store the DAGs on GIT, and configure all components to pull from that GIT repo. Just as today, the scheduler should pull every X seconds from that repository. When you want to add new DAGs, you simply need to push it to that repo. It will also allow us to have some version control over DAGs, which could be useful for keeping cleaner code and order. -- This message was sent by Atlassian Jira (v8.3.4#803005)