jens-scheffler-bosch commented on issue #30327:
URL: https://github.com/apache/airflow/issues/30327#issuecomment-1700626246

   I tested this bug report with the provided example on Airflow 2.7.0 and 
latest main and can confirm this "strange" behavior. Made some regression and 
can confirm that even when `catchup=False` is set, an execution of a manual run 
from a DAG with a custom TimeTable adds scheduled jobs filling the gap. In 
total with my tests the scheduling behavior looks inconsistent like below:
   
   Scheduling Expression | Initial Run of Last Time Period | Filling Gaps | 
Notes
   --- | --- | --- | ---
   Custom TImetable as example `AfterWorkdayTimetable` | No. Current time 
period is not started automatically | Yes. When a manual with a logical date in 
the past is executed, occurrences after this are started and fill the gaps. | 
This is actually unexpected and inconsistent to other default timetables.
   Classic cron expression `"0 0 * * *"` | Yes. An initial run for the current 
period is made | No. | This is how I expected it as well for a custom timetable.
   Usage of `CronTriggerTimetable` | Yes. An initial run for the current period 
is made | No. | This is how I expected it as well for a custom timetable.
   Usage of `EventsTimetable` | Ups, all events defined in the past were 
executed. | n/a | This was a surprise. Whereas I would see this also as a 
feature but all defined events in the past were run even when `catchup=False` 
was set. Might be another bug? Or explicity feature? Could not find any 
mentioning in the docs, so rather documentation flaw of bug.
   Usage of `datetime.timedelta()` | Yes. An initial run for the current period 
is made | ? Another ups. A second run was scheduled after the manual trigger. | 
This was also rather unexpected. Can be repeated. Whenever a run with a logical 
date in the past is triggered, one additional scheduled run based on current 
time is started as well.
   
   I tried to walk through the code in `airflow/jobs/scheduler_job_runner.py` 
which I assume is the core part of logic where this is implemented. But after 
drilling into the code I was lost :-( and not able to pin down the root error. 
I fear an scheduler expert is needed here for investigation. (@ashb ?)
   I assume it is an artifact related to the AIP-39 integration.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to