Re: Setting Connections via Env Vars

2016-10-01 Thread Laura Lorenz
We (Industry Dive) also use environment variable based configuration to configure our airflow system. Our particular case is that we use docker-compose for configuration management between environments of airflow services we deploy as docker containers, which can easily forward environment

Re: Airflow bugs but stays running

2016-10-01 Thread siddharth anand
Hi Renaud, I've never encountered this issue though I do run postgres & LocalExecutor and am running 1.7.1.3 in all of my environments. I'm running on master on my local dev machine. I changed the valid sql_alchemy_conn = postgresql://siddharth@localhost:5432/airflow to the invalid

Re: ExternalTaskSensor cases run time exception when performing the query to airflow DB

2016-10-01 Thread siddharth anand
Hila, I commented just now. -s On Sun, Sep 25, 2016 at 10:11 PM, הילה ויזן wrote: > Hi guys, > I posted this issue 2 weeks ago. > ExternalTaskSensor fails in each dag run - query to airflow DB fails. > > maybe one of you can help? > >

Re: Next Airflow meet-up

2016-10-01 Thread Alex Van Boxel
Hey guys, about the date. There is good chance I'm in SF for summit (14-15 November), I could try to extend it a day (so including November 16th). So if you're looking for a date, think about November 16th (I'll volunteer for a talk then). Would be great. Thanks. On Sat, Oct 1, 2016 at 8:51 AM

Re: Setting Connections via Env Vars

2016-10-01 Thread George Leslie-Waksman
I would absolutely prefer to see them in the UI, and there are some annoying details in the way that some hooks use env vars in non-standard ways (I'm looking at you S3Hook) or have extra parameters that are hard (impossible?) to include via the environment. It might be nice if the UI pulled

Re: Next Airflow meet-up

2016-10-01 Thread siddharth anand
SGTM. -s On Fri, Sep 30, 2016 at 10:18 PM, George Leslie-Waksman < geo...@cloverhealth.com> wrote: > I'll start the ball rolling at Clover and try to lock in a date with plenty > of advance notice. > > I'll probably aim for early December (first week perhaps) to avoid > Thanksgiving and the

Re: Setting Connections via Env Vars

2016-10-01 Thread siddharth anand
I see. Thanks for the explanation. Then, wouldn't you like Airflow's connections views to read, edit, and delete connection information, whether that connection information is stored in the db or in the OS environment? Currently, OS env-provided connections are not exposed by the web app! Sure,