yuseok89 commented on issue #21867: URL: https://github.com/apache/airflow/issues/21867#issuecomment-6553114766
Since Airflow 3.4, a task group can be run again as a unit with a loop. Make the group a loop whose final task uses `trigger_rule="all_done"` and returns whether the iteration succeeded, and stop the loop with `until` once it did. A failed iteration then runs the whole group again, up to `max_iterations`, and the Dag run succeeds once an iteration succeeds. I documented this with an example in #76187. A delay between attempts, like the hour in the original request, also works by starting the group with a deferrable sensor that waits only from the second iteration on. This looks to cover the cases in this issue, including a condition other than "any task failed", since `until` can check anything the final task returns. A group retried this way cannot be placed inside another one or inside a mapped task group, though. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
