[GitHub] [airflow] uranusjr commented on issue #19058: Manual dagrun logs to last scheduled dagruns log files

2021-11-16 Thread GitBox
uranusjr commented on issue #19058: URL: https://github.com/apache/airflow/issues/19058#issuecomment-970475723 Tracking the rest of the work in #19625. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [airflow] uranusjr commented on issue #19058: Manual dagrun logs to last scheduled dagruns log files

2021-11-02 Thread GitBox
uranusjr commented on issue #19058: URL: https://github.com/apache/airflow/issues/19058#issuecomment-957160954 `data_interval_start` is the start of the DAG run's data interval, which `logical_date` is a date that identifies that DAG run. They are the same for runs scheduled via CRON and `

[GitHub] [airflow] uranusjr commented on issue #19058: Manual dagrun logs to last scheduled dagruns log files

2021-11-02 Thread GitBox
uranusjr commented on issue #19058: URL: https://github.com/apache/airflow/issues/19058#issuecomment-957160954 `data_interval_start` is the start of the DAG run's data interval, which `logical_date` is a date that identifies that DAG run. They are the same for runs scheduled via CRON and `

[GitHub] [airflow] uranusjr commented on issue #19058: Manual dagrun logs to last scheduled dagruns log files

2021-11-02 Thread GitBox
uranusjr commented on issue #19058: URL: https://github.com/apache/airflow/issues/19058#issuecomment-957160954 `data_interval_start` is the start of the DAG run's data interval, which `logical_date` is a date that identifies that DAG run. They are the same for runs scheduled via CRON and `

[GitHub] [airflow] uranusjr commented on issue #19058: Manual dagrun logs to last scheduled dagruns log files

2021-10-21 Thread GitBox
uranusjr commented on issue #19058: URL: https://github.com/apache/airflow/issues/19058#issuecomment-948525543 The problem is we need to finish all backfilling before we change the default filename format (or before the user changes the custom format based on our recommendation), otherwise

[GitHub] [airflow] uranusjr commented on issue #19058: Manual dagrun logs to last scheduled dagruns log files

2021-10-19 Thread GitBox
uranusjr commented on issue #19058: URL: https://github.com/apache/airflow/issues/19058#issuecomment-947327336 > add a new column `log_filename` on `TaskInstance` that's populated exactly once when the TI is run, to decouple the filenamne derivation logic. But considering how poor th

[GitHub] [airflow] uranusjr commented on issue #19058: Manual dagrun logs to last scheduled dagruns log files

2021-10-19 Thread GitBox
uranusjr commented on issue #19058: URL: https://github.com/apache/airflow/issues/19058#issuecomment-947023469 > I believe we could simply ban using `ts` in the names of the logs. That would mean some additional analysis of the custom logging specification and failing airflow if we detect

[GitHub] [airflow] uranusjr commented on issue #19058: Manual dagrun logs to last scheduled dagruns log files

2021-10-19 Thread GitBox
uranusjr commented on issue #19058: URL: https://github.com/apache/airflow/issues/19058#issuecomment-946580267 OK, I think I know what's going on. The log handler uses a template to render the log's filename, which (by default) is `{{ ti.dag_id }}/{{ ti.task_id }}/{{ ts }}/{{ try_number }}