Re: [DISCUSS] AIP-55 Rule-based timetable with logical composition

2023-03-31 Thread Jarek Potiuk
> 1) *Calendars are dynamic*.Special events cannot be avoided: 911 - financial markets and air flights were cancelled for a while in the US, country wide protests due to law changes in France right now, etc. These are immediate changes to "the calendar". We need to be able to push calendar

Re: [DISCUSS] AIP-55 Rule-based timetable with logical composition

2023-03-31 Thread David Robinson
I also like this feature suggestion and think it steps in the right direction. This idea is crucial for businesses who schedule their data pipelines and other actions based on Calendars. We use multiple calendars depending on the countries or even regions in which we operate. In addition to the

Re: [DISCUSS] AIP-55 Rule-based timetable with logical composition

2023-03-31 Thread Jarek Potiuk
Yep. I also think it's a good addition. Though again - I don't think AIP is something that we need here, simply a PR where it could be iterated on, should be enough IMHO. It's **just** another timetable to add, we are not changing any architectural decision here. I'd also separate out the

Re: [DISCUSS] AIP-55 Rule-based timetable with logical composition

2023-03-31 Thread Mehta, Shubham
Hi Malthe I really like this feature suggestion, especially since it's crucial for customers who are transitioning from Control-M or Autosys [1] to Airflow. Implementing this feature has the potential to bring Airflow's scheduling capabilities on par with these tools. > I actually added some

Re: [DISCUSS] AIP-55 Rule-based timetable with logical composition

2023-03-30 Thread Malthe Borch
> I think an interesting topic to discuss for this project would be some kind > of UI based date/calendar picker to help users construct these logical > compositions. Something like `days("D1", "D2", "THU-SAT", "4>", "L1")` is > quite inscrutable. Agreed – and really, the syntax is something

Re: [DISCUSS] AIP-55 Rule-based timetable with logical composition

2023-03-27 Thread Oliveira, Niko
Jarek Potiuk Sent: Sunday, March 26, 2023 2:52:16 PM To: dev@airflow.apache.org Subject: RE: [EXTERNAL][DISCUSS] AIP-55 Rule-based timetable with logical composition CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the s

Re: [DISCUSS] AIP-55 Rule-based timetable with logical composition

2023-03-26 Thread Jarek Potiuk
I am not sure if it needs an AIP - just PR implementing it and discussion there should be more than enough IMHO. On Thu, Mar 23, 2023 at 10:51 AM Malthe wrote: > > This AIP comes out of a previous discussion on skipping tasks based on > a rule-based schedule, e.g. excluding holidays except if

[DISCUSS] AIP-55 Rule-based timetable with logical composition

2023-03-23 Thread Malthe
This AIP comes out of a previous discussion on skipping tasks based on a rule-based schedule, e.g. excluding holidays except if it's Monday. The central idea is to define a schedule based on logical composition (and, or, not) – using a small number of primitives.