Re: When to use pools?

2016-06-21 Thread Dan Davydov
Assuming you are using local/sequential executors, the backfill pool would be used. On Mon, Jun 20, 2016 at 10:47 PM, harish singh wrote: > hmm.. Thanks Lance. I mentioned about pool for 'backfill' is because I saw > that being a part 'default_args' airflow example. > > Chris/Dan/Bolke/Jeremiah/

Re: dagrun_timeout not honoured?

2016-06-21 Thread Ben Tallman
Already there, twice, but one with a pull request... https://github.com/apache/incubator-airflow/pull/1601 On Tue, Jun 21, 2016 at 4:53 PM Ben Tallman wrote: > Maxime - > > Wish I did have time... BUT, I can say that SLA timeouts will fail (and > error) on any dag with schedule=None because the

Re: dagrun_timeout not honoured?

2016-06-21 Thread Ben Tallman
Maxime - Wish I did have time... BUT, I can say that SLA timeouts will fail (and error) on any dag with schedule=None because they depend on the Now() being less than the next scheduled date (None)... dttm = dag.following_schedule(dttm) while dttm < datetime.now(): I will file a bug... On Tue,

Re: dagrun_timeout not honoured?

2016-06-21 Thread Maxime Beauchemin
A tangent here: for people who have the knowledge (and a bit of time on their hand), providing a failing unit test can help the core committers with an easy way to jump in to help. I always wonder whether I'll be able reproduce the bug, is it version specific? environment specific? is it based on

Re: dagrun_timeout not honoured?

2016-06-21 Thread Ben Tallman
We have seen this too. Running 1.7.0 with Celery, neither DAG timeout nor individual task sla's seem to be honored. In truth, we haven't done a lot of testing, as it is more important that we get our overall ETL migrated with workarounds. However, we will be digging in at some point for greater cl

Re: Airflow logo; override templates

2016-06-21 Thread Chris Riccomini
Hey Tatsiana, Good question--I'd like to know as well. We've been committing/cherry picking an internal commit that stylizes Airflow according to our UI needs (logo and color scheme). It'd be nice if there was a pluggable way to do this. Cheers, Chris On Tue, Jun 21, 2016 at 6:29 AM, Tatsiana K

Airflow logo; override templates

2016-06-21 Thread Tatsiana Klionskaya
Hello , Is there a way to change the logo of Airflow using the plugins? Is there a way to override admin/master.html, airflow/master.html templates using the plugins? Thank you in advance. -Tatsiana-