Hi Andrew, I've ran into the same problem and I found out that the best way to do this is to use a dag_factory that takes your parameter and builds a dag for each lob in the "list".
This way you'll have one dag per lob, and they will run independently as I believe its your intention. This solution will define which dags are active or inactive not at DAG runtime, but at scheduler boot time. The problem you may have or not, is if the lob "list" is somehow dynamic, which means that your dags will be active or inactive depending on the lob being in the "list" at the time the scheduler instantiates your dag and not necessarily at the time your dag runs. I hope I've helped, I believe this is a point where best practices are still in some kind of grey area and is definitely something airflow should have one clear way of doing. On Tue, Mar 21, 2017 at 4:06 PM Andrew Maguire <[email protected]> wrote: > Sorry if this is not the right place for this. > > I'm wondering if anyone might be able to help me with this question? > > > http://stackoverflow.com/questions/42930221/best-way-to-loop-through-parameters-in-airflow > > Feel free to ignore if this is not the right place to try find help. > > Loving airflow btw! > > Cheers, > Andy >
