Interesting. You say that "A Triggered DAG has the execution date of when it is actual run or it’s given time to execute." In my case, the triggering dag has cron schedule of... 0 5 1,3-31 * * <http://airflowetl.ucera.local:8080/admin/dagrun/?flt2_dag_id_equals=hph_clarity_etl_trigger>
and the triggered dag has cron schedule... None <http://airflowetl.ucera.local:8080/admin/dagrun/?flt2_dag_id_equals=hph_clarity_etl> So the triggered dag has no given time to automatically execute. Could you explain a bit more about how these schedules and the TriggerDagRunOperator are interacting to create a logged time of +1 day from the triggering dag's last run? On Fri, Mar 6, 2020 at 5:06 AM Shaw, Damian P. < [email protected]> wrote: > Firstly, you can use other times other than UTC, you set the start date of > the dag to a different time zone: > https://airflow.apache.org/docs/stable/timezone.html#time-zone-aware-dags . > Note that the UI and ExecutionDate will still be in UTC, if you need to get > the ExecutionDate in your local timezone this is currently best achieved > through Macros: > https://stackoverflow.com/questions/44855949/make-custom-airflow-macros-expand-other-macros/54922776#54922776 > . I run dozens of DAGs in about more than 5 different timezones this way. > > > > Secondly the execution date of Triggered DAGs has a different logic to > Scheduled DAGs (I’m crossing my fingers this inconsistency is resolved in > Airflow 2.0 by getting rid of the weird Scheduled DAG logic): > > * A Triggered DAG has the execution date of when it is actual run or it’s > given time to execute. For example say you set it to run at 2020-03-06 > 12:00:00 then that is it’s execution date. > > * A Scheduled DAG has an execution date that subscribes to the logic that > the DAG is processing a “period” of data. So for example if you schedle it > to run at midday every day then the run at 2020-03-06 12:00:00 considers > that period of scheduling to be from 2020-03-05 12:00:00 (inclusive) to > 2020-03-06 12:00:00 (not inclusive), it then gives you an execution date of > 2020-03-05 12:00:00 > > > > As you can see even though their actual execution time is the same, their > execution date is 1 day apart. > > > > A Triggered DAG can not provide the same logic as a Scheduled DAG because > it has no schedule to determine the “period”. I have created Macros that > gives me the same execution date regardless of whether I am using Scheduled > or Triggered DAGs, you can use the above SO answer as a template if you > want to do the same, and the thing to remember is that for a Scheduled DAG > the next execution date is equivelent to the Triggered DAG execution date. > > > > Hope this helps > > Damian > > > > > > > > *From:* Reed Villanueva <[email protected]> > *Sent:* Thursday, March 5, 2020 21:35 > *To:* [email protected] > *Subject:* Dag triggered via TriggerDagRunOperator logs start date as 1 > day ahead > > > > Seeing a weird problem where when running a dag on cron schedule > > *0 5 1,3-31 * ** > <http://airflowetl.ucera.local:8080/admin/dagrun/?flt2_dag_id_equals=hph_clarity_etl_trigger> > > > (note I am still using UTC time (since I do not believe that I can change > it in v1.10.7)) that ends by running a TriggerDagRunOperator task. > > When the "triggering" dag runs, the Last Run date shows the correct date > and is reflected in the run_id. Eg. > > 2020-03-04 05:00 > <http://airflowetl.ucera.local:8080/admin/airflow/graph?dag_id=hph_clarity_etl_trigger&execution_date=2020-03-04+05%3A00%3A00%2B00%3A00> > > > However, the dag that gets triggered shows the Last Run date as 1 day > ahead and this unexpected behavior is also reflected in the triggered dag's > run_id > > 2020-03-05 05:00 > <http://airflowetl.ucera.local:8080/admin/airflow/graph?dag_id=hph_clarity_etl&execution_date=2020-03-05+05%3A00%3A47.975014%2B00%3A00> > > > yet the triggering dag only takes about 1min. to run. > > > > Does anyone know what could be happening here? > > Is there a time difference that I am not recognizing here (note that I am > not living in a UTC time zone (I am in HST and the cron is intended to run > the triggering dag at 7:00PM each night excluding the 1st of each month))? > > > This electronic message is intended only for the named > recipient, and may contain information that is confidential or > privileged. If you are not the intended recipient, you are > hereby notified that any disclosure, copying, distribution or > use of the contents of this message is strictly prohibited. If > you have received this message in error or are not the named > recipient, please notify us immediately by contacting the > sender at the electronic mail address noted above, and delete > and destroy all copies of this message. Thank you. > > > > > ============================================================================== > Please access the attached hyperlink for an important electronic > communications disclaimer: > http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html > > ============================================================================== > -- This electronic message is intended only for the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.
