> > This would be great. I agree that if there were ready-to-use > timetables with a composable behavior then very few users will need to > write custom timetables which is a good thing. > > Cool!
> > I do not think the current interface is "too complex". Not at all. But I > think that it is targeted to a different audience than Malthe and Bas talk > about. It is addressed for "power users" - not only because it requires > deep understanding of Airflow scheduling internals and optimizations but > also, because it requires "admin" rights to develop, test and install it. > Regular users. who are Dag authors cannot create new Timetables. This is > mostly because of security. The "regular users" need to convince the > admins to do so. And yes I am talking about the important segment of our > users where you have professional admins/devops configuring Airflow and DAG > authors who just write DAGs. I think this is the most interesting and > biggest segment of our users to be honest. We should always think about > this segment of our users first IMHO. > > Well I think they're too complex! > > And I actually elaborated quite a bit on that in this correspondence – > including giving a concrete proposal which you did not consider or > mention. But I'm happy to be proven wrong. Perhaps it is only me who > thinks the interface is wrong. Let us see an example implementation of > "-2 day of every month" or "every day after work", either using a > declarative specification built on top of some composable timetable, > or as a direct implementation. > > Precisely. I think we should implement those and then we can see if they can be simplified. I think if we have a few customized timetables including comprehensive unit tests, we will be able to see if we can simplify the whole interface for Airflow 3. But having those few predefined timetables and the unit tests for those - will be of a great help when we will want to simplify it IMHO. Then for Airflow 3 (which is still months away if not years) we will be in a much better position to make even breaking changes. > > - Composability. It should be possible to use and/or operators in a > nested fashion to compose any timetable (e.g. every Friday OR every > last weekday AND every day at 6pm). > - Intervals. It should be possible to define a timetable which changes > over time, as non-overlapping intervals (e.g. one of each year). > - Exclude. It should be possible to exclude certain days (i.e. holidays). > By all means : - most of those were already mentioned as "future work" in https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-39+Richer+scheduler_interval - composability, intervals. I think we should just follow up what has been planned there. > I think this is a good starting point and it would allow users to meet > most of their scheduling needs. > > Cheers >
