On Wed, Feb 23 2022 at 14:13:43 +0100, Bas Harenslak
<b...@astronomer.io.INVALID> wrote:
catchup must be implemented which I feel should be generic to all
Timetables and not left up to a developer.
Catchup is exposed to the Timetable interface to enable this use case
<https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-39+Richer+scheduler_interval#AIP39Richerscheduler_interval-FutureEnhancements>
*"Rollup", an alternative to backfill*
This covers a few use-cases, but once we have an explicit start and
end of interval defined it Is desirable to be able to re-run the same
(for example) daily dag for a longer period.
This could be an alternative to backfill, where the Trigger interface
in the UI could show optional date range inputs letting the user say
"run this dag for a 1 month period".
This behaviour also leads to a new alternative method of doing
catchup: if three dag runs are missed, rather than creating three
individual daily runs (for example) we could instead create one DAG
run with a three-day data interval. This would be opt-in on a per-dag
basis, as it depends entirely on how the task is written
(The last paragraph there specifically)
