Re: Creating dynamic pool from task

2018-09-27 Thread Taylor Edmiston
hitepages.com> > > > wrote: > > > > > >> What I am doing is very similar. However I am including the DagRun's > id > > in > > >> the pool name to make it unique, as I need to make sure every run gets > > its > > >> own pool. I am getti

Re: Creating dynamic pool from task

2018-09-25 Thread Chris Palmer
te methods or templates. How do you make sure each > >> run has it's own pool? > >> > >> > >> Thanks, > >> > >> Dávid Szakállas > >> Software Engineer | Whitepages Data Services > >> > >> >

Re: Creating dynamic pool from task

2018-09-21 Thread Chris Palmer
un has it's own pool? > > > Thanks, > > Dávid Szakállas > Software Engineer | Whitepages Data Services > > > From: Taylor Edmiston > Sent: Thursday, September 20, 2018 6:17:05 PM > To: dev@airflow.incubator.apache.org > Subject

Re: Creating dynamic pool from task

2018-09-20 Thread David Szakallas
has it's own pool? Thanks, Dávid Szakállas Software Engineer | Whitepages Data Services From: Taylor Edmiston Sent: Thursday, September 20, 2018 6:17:05 PM To: dev@airflow.incubator.apache.org Subject: Re: Creating dynamic pool from task I've done something

Re: Creating dynamic pool from task

2018-09-20 Thread Taylor Edmiston
I've done something similar. I have a task at the front of the DAG that ensures the connection pool exists and creates the pool if it doesn't. I've pasted my code below. This runs in a for loop that creates one DAG per iteration each with its own pool. Then I pass the pool name into the