Specify volume for Kubernetes Executor created workers

2019-08-31 Thread Sourabh Bajaj
Hi, I'm trying to run Airflow with the new Kubernetes Executor and was trying to have each worker that is started attach a persistant volume they can use for saving intermediate results. I could not find how to pass this in the config when this is neither dags or logs. One workaround I had was to

Re: Using IAM auth token for sql_alchemy

2019-08-22 Thread Sourabh Bajaj
e at process (scheduler, > > webserver) start up, so that wouldn't work for IAM creds, sadly. > > > > To support this probably needs some deeper changes in Airflow - but I don't > > have an immediate idea what they might need to be. It would be nice thou

Using IAM auth token for sql_alchemy

2019-08-21 Thread Sourabh Bajaj
Hi, I'm trying to use IAM created auth tokens for connecting to the airflow database. One question I had was: How often does sql_alchemy_conn_cmd get called? so want to figure out the lifetime of a connection. Best, Sourabh