Re: Bit confused about start_date and schedule_interval related to daily/weekly DAG

2018-04-18 Thread Ruiqin Yang
Hi Kyle, The execution_date of the DAG run will always be lagged one day for your daily DAG and one week for your weekly DAG. Under the hood, airflow will calculate the execution_date and next execution_date of the task, and only schedule the task when the current timestamp is bigger than the

Bit confused about start_date and schedule_interval related to daily/weekly DAG

2018-04-18 Thread Kyle Hamlin
I'm a bit confused with how the scheduler catches up in relation to start_date and schedule_interval. I have one dag that runs hourly: dag = DAG( dag_id='hourly_dag', start_date=days_ago(1), schedule_interval='@hourly', default_args=ARGS) When I start this DAG fresh it will catch