Re: Cron expression with depends_on_past causing no TIs to be generated

2017-01-14 Thread Sumit Maheshwari
Thanks a lot Bolke for looking into this issue and raising PR in such a short time. On Sat, Jan 14, 2017 at 8:52 PM, Bolke de Bruin wrote: > PR is out. Should fix your issue and fix a bug that was introduced with > the catchup feature. > >

Re: Cron expression with depends_on_past causing no TIs to be generated

2017-01-14 Thread Bolke de Bruin
PR is out. Should fix your issue and fix a bug that was introduced with the catchup feature. https://github.com/apache/incubator-airflow/pull/1994 Bolke > On 14 Jan 2017, at 14:22, Bolke de Bruin wrote: > > You will

Re: Cron expression with depends_on_past causing no TIs to be generated

2017-01-14 Thread Bolke de Bruin
You will need to align your start_date with the cron interval. In this case your start date should include 30 minutes. This is expected behaviour in 1.7.0. and 1.7.1. In master we are auto aligning the start_date with the interval in the scheduler. However, the dependency checker doesn’t do

Re: Airflow 1.8.0 alpha 5

2017-01-14 Thread Bolke de Bruin
Hi Alex, Don’t worry it is a switch and thus optional. This is also present in 1.7.1 and is actively used. The current state of master is a regression in that respect. Bolke > On 14 Jan 2017, at 10:13, Alex Van Boxel wrote: > > Hey Bolke, I've missid this requirement:

Cron expression with depends_on_past causing no TIs to be generated

2017-01-14 Thread Sumit Maheshwari
Found that on our current prod version (1.7.0) and master as well, scheduler doesn't fire TIs if *scheduler_interval* is a cron expression and *depends_on_past* is True. The very simple DAG I used for testing can be found here: http://pastebin.com/wPnFjRwD However it creates new DAG runs as per

Re: Airflow 1.8.0 alpha 5

2017-01-14 Thread Alex Van Boxel
Hey Bolke, I've missid this requirement: daemonizing the webserver?! Don't do this by default. That would brake running on Kubernetes or in Docker. Make it optional with a flag! On Fri, Jan 13, 2017 at 8:48 PM Bolke de Bruin wrote: > Dear All, > > I have made Airflow 1.8.0