interesting, thanks Bolke for giving me this idea! On Tue, Jan 17, 2017 at 11:29 AM, Bolke de Bruin <bdbr...@gmail.com> wrote:
> Well yes I guess so. You can actually create pools from a DAG definition > as well (I personally don’t like that for operational security reasons), > but yes this works. So could verify if the pool exists for a certain DAG > and create it if it doesn’t. > > Bolke > > > On 17 Jan 2017, at 17:26, Boris Tyukin <bo...@boristyukin.com> wrote: > > > > interesting but I have 200 DAGs like that - I generate them > > programmatically. Does it mean I need a different pool for each one of > > them? Can I define pools not from UI? that would be painful :) > > > > On Tue, Jan 17, 2017 at 10:22 AM, Bolke de Bruin <bdbr...@gmail.com> > wrote: > > > >> You could use a pool, if you are fine with the “second” dag run being > run > >> later. > >> > >> > >>> On 17 Jan 2017, at 16:18, Boris Tyukin <bo...@boristyukin.com> wrote: > >>> > >>> hello, > >>> > >>> I would like to prevent a DAG from scheduling / running concurrently > >>> whether by scheduler or if triggered externally (via airflow > trigger_dag > >> or > >>> UI). > >>> > >>> I tried to set max_active_runs to 1 but i did not work for externally > >>> triggered case - airflow would still go ahead and execute multiple dag > >> runs. > >>> > >>> I see this is done intentionally in dagrun_exists_dep.py: > >>> > >>> running_dagruns = DagRun.find( > >>> dag_id=dag.dag_id, > >>> state=State.RUNNING, > >>> *external_trigger=False,* > >>> session=session > >>> ) > >>> Is there any other way of forcing only one DAG run at the time? > >>> > >>> I am aware of depends_on_past=True but I use PythonBranch operators and > >>> that setting does not make sense in my case. > >> > >> > >
