shmcsensei commented on issue #27399:
URL: https://github.com/apache/airflow/issues/27399#issuecomment-1640860585
I've been testing the following change to fix this issue seems to be
working. Its similar to what has been suggested:
```python
def next_dagrun_info(
self,
*,
last_automated_data_interval: Optional[DataInterval],
restriction: TimeRestriction,
) -> Optional[DagRunInfo]:
if restriction.catchup:
if last_automated_data_interval is None:
if restriction.earliest is None:
return None
next_start_time = self._align_to_next(restriction.earliest)
else:
next_start_time =
self._get_next(last_automated_data_interval.end)
else:
current_time = DateTime.utcnow()
if restriction.earliest is not None and current_time <
restriction.earliest:
next_start_time = self._align_to_next(restriction.earliest)
else:
if last_automated_data_interval is None:
# The DAG has never run before.
next_start_time = self._align_to_next(current_time)
elif last_automated_data_interval.end ==
self._align_to_prev(current_time):
# The last run is the same as the previously scheduled run.
Everything is working fine.
next_start_time = self._align_to_next(current_time)
else:
# The last run is NOT the same as the previously scheduled
run, we must have skipped it.
next_start_time = self._align_to_prev(current_time)
LOGGER.warn('DAG skipped previous schedule, rescheduling.
last_automated_data_interval.end:"%s" next_start_time:"%s"',
last_automated_data_interval.end, next_start_time)
if restriction.latest is not None and restriction.latest <
next_start_time:
return None
return DagRunInfo.exact(next_start_time)
```
Specifically this addition:
```python
if last_automated_data_interval is None:
# The DAG has never run before.
next_start_time = self._align_to_next(current_time)
elif last_automated_data_interval.end == self._align_to_prev(current_time):
# The last run is the same as the previously scheduled run. Everything
is working fine.
next_start_time = self._align_to_next(current_time)
else:
# The last run is NOT the same as the previously scheduled run, we must
have skipped it.
next_start_time = self._align_to_prev(current_time)
LOGGER.warn('DAG skipped previous schedule, rescheduling.
last_automated_data_interval.end:"%s" next_start_time:"%s"',
last_automated_data_interval.end, next_start_time)
```
I added the logging so I could monitor it and alert if it happens, just in
case. Not sure if this could be done better though, I'm open to suggestions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]