Re: Scheduler won't schedule past minimum end_date of tasks

2018-02-22 Thread Chris Palmer
I don't think there is an issue with TIDeps, there is already a RunnableExecDateDep that covers that end dates of tasks. Here is my toy example that I used to test this behavior - https://paste

Re: Scheduler won't schedule past minimum end_date of tasks

2018-02-22 Thread Ash Berlin-Taylor
That does sound like a bug, and I would have expected, as you did, that not specifying an end_date on some tasks means those tasks should run for ever. Changes that probably need making is that a task end_date of None on a task should me "greater" than other task dates in/around the lines you li

Scheduler won't schedule past minimum end_date of tasks

2018-02-21 Thread Chris Palmer
I was very surprised to find that if you set an end_date on any of the tasks in a DAG, that the scheduler won't create DagRuns after the minimum end_date of tasks. The code that does this is the 6 or so lines starting here - https://github.com/apache/incubator-airflow/blob/master/airflow/jobs.py#L8