Well yes I guess so. You can actually create pools from a DAG definition as well (I personally don’t like that for operational security reasons), but yes this works. So could verify if the pool exists for a certain DAG and create it if it doesn’t.
Bolke > On 17 Jan 2017, at 17:26, Boris Tyukin <bo...@boristyukin.com> wrote: > > interesting but I have 200 DAGs like that - I generate them > programmatically. Does it mean I need a different pool for each one of > them? Can I define pools not from UI? that would be painful :) > > On Tue, Jan 17, 2017 at 10:22 AM, Bolke de Bruin <bdbr...@gmail.com> wrote: > >> You could use a pool, if you are fine with the “second” dag run being run >> later. >> >> >>> On 17 Jan 2017, at 16:18, Boris Tyukin <bo...@boristyukin.com> wrote: >>> >>> hello, >>> >>> I would like to prevent a DAG from scheduling / running concurrently >>> whether by scheduler or if triggered externally (via airflow trigger_dag >> or >>> UI). >>> >>> I tried to set max_active_runs to 1 but i did not work for externally >>> triggered case - airflow would still go ahead and execute multiple dag >> runs. >>> >>> I see this is done intentionally in dagrun_exists_dep.py: >>> >>> running_dagruns = DagRun.find( >>> dag_id=dag.dag_id, >>> state=State.RUNNING, >>> *external_trigger=False,* >>> session=session >>> ) >>> Is there any other way of forcing only one DAG run at the time? >>> >>> I am aware of depends_on_past=True but I use PythonBranch operators and >>> that setting does not make sense in my case. >> >>
