I know catchup_by=true by default, but I think when user run example dag, their just want to test how Airflow run, and thier would stop it manually.
But if user use start_date=airflow.utils.dates.days_ago(2) as their production dag would cause unexpected behavior. IMO, if we use fixed date in example dag, dag would backfill, but user could stop it manually, and their would know that "startdate control when dag start", and user will use fixed date to their new dag(template from example dag) Best wish. -- Jiajie ________________________________ From: Deng Xiaodong <[email protected]> Sent: Friday, May 10, 2019 15:50 To: [email protected] Subject: Re: [DISCUSS] Change all example_dags's start_date to specific datetime Please note that catchup_by_default is True by default. Imagine what will happen if you set these start_date to fixed dates ;-) XD On Fri, May 10, 2019 at 3:34 PM Jiajie Zhong <[email protected]> wrote: > Hi Airflowers: > I start this thread proposal to change all example_dags's start_date > to specific datetime, including airflow/example_dag and > airflow/contrib/example_dags. > I proposal that because in > https://airflow.apache.org/faq.html#what-s-the-deal-with-start-date > > > We recommend against using dynamic values as start_date, especially > datetime.now() as it can be quite confusing. The task is triggered once the > period closes, and in theory an @hourly DAG would never get to an hour > after now as now() moves along. > > we suggest to set start_date to specific datetime, but all example_dags in > code base are use dynamic values like `airflow.utils.dates.days_ago(2)` or > `datetime.utcnow()`. Most of Airflow starter will use example dag as they > template to create their own dag, so I think should change them in correct > values. > > But if we use a specific datetime in example dag, for example 2019-01-01, > starter will use them as template and confuse "Why my dag start run in > 2019-01-01?". > > So, should we change start_date? > > Best wish. > -- Jiajie >
