vladimirbinshtok commented on PR #53083: URL: https://github.com/apache/airflow/pull/53083#issuecomment-3065309123
@o-nikolas This retry was invented to allow a separate retry mechanism on infrastructure errors, which can make sense if the triggered business logic is not idempotent (so the developer turns off Airflow retry); otherwise, Airflow retry can do the same job. This retry mechanism was invented before deferrable mode (https://github.com/apache/airflow/pull/25413) and worked as expected, while `@AwsBaseHook.retry(should_retry_eni)` was wrapping the start task function. While the migration to support deferrable mode, this retry was deleted (https://github.com/apache/airflow/pull/32104), and the retry mechanism stopped working. Regarding the breaking change, the retry is not working since version `apache-airflow-providers-amazon` 8.6.0, which was added in Airflow 2.7.1. Reinventing the retry with deferrable mode is a complicated task, as it requires a separate retry variable to be passed around the system. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
