Airflow 1.9.1 expected release?

2018-04-18 Thread Zsolt Tóth
Hey all, is there a planned (approximate) release date for Airflow 1.9.1? Best, Zsolt

Re: Airflow Deployment tools

2017-11-15 Thread Zsolt Tóth
We are also using Ansible for: - Installing/upgrading/configuring Airflow (there are several airflow roles on git) - Deploying the pipelines - Restarting Airflow webserver/scheduler It would be great to have Airflow manageable from Hadoop cluster managers (Cloudera Manager, Ambari). For this a

Airflow webserver forgets dags on refresh

2017-07-18 Thread Zsolt Tóth
Hi, I'm using Airflow 1.8.1, but also tried with 1.8.0 and 1.8.2-rc1, with LocalExecutor, on a single node. When I click Refresh on the UI (either in the Dag's page, or in the main DAG list), there is a good chance that airflow somehow does not find the DAG anymore and displays the stacktrace

Re: Executing Dags atomically

2017-06-02 Thread Zsolt Tóth
com>: > Try an ExternalTaskSensor > > Thanks, > > Scott > > On Jun 2, 2017, 3:28 AM -0700, Zsolt Tóth <toth.zsolt@gmail.com>, > wrote: > > Hi, > > > > is there a way to execute all tasks of a dag without interruption? Right > > now th

Executing Dags atomically

2017-06-02 Thread Zsolt Tóth
Hi, is there a way to execute all tasks of a dag without interruption? Right now the scheduler jumps between dags, and eventually they will all finish. What I want is to force the scheduler to finish a dag completely, and jump to the next one afterwards. I think I could set all tasks of the dag

Re: Subdag failure

2017-06-01 Thread Zsolt Tóth
is a bit confusing and it could help to make `--local` and `--raw` > > more "private", as in for system/internal use only. > > > > I also just noticed that the `--local` for `airflow run` is wrong in the > > docs as it's re-using the backfill help text, I'll send a PR to