Hi AirFlow Developer, I am Dr. Soma Dhavala, ML architect. At EkStep <https://ekstep.org/>, a non-profit organisation, we are developing an open source ML workbench and we are using AirFlow as one of the core components, on the top of which are writing an application layer. The idea is to make this tool available to millions of users who use our SunBird <http://www.sunbird.org/about/> platform
We compose DAGs programmatically - that is an agent auto-composes a DAG based on a specification (i.e., DAGS are not handcrafted but are generated based a on template), so we dont have any python script that literally constructs a DAG object. So it was less than ideal for us to drop .py files into the DAG folder. An ability to save/write DAG objects directly to the DB or make them available to the DagBag would ideal. Right now, we drop a py script that will load pickled DAG objects and makes them available to the DagBag upon its instantiation. The work around we are using can be seen here https://github.com/dhavala/airflow-exp I am wondering if you have any plans to inject DAG objects via methods available on the "dag" class or something similar. thanks, -soma