0x26res opened a new issue, #39607: URL: https://github.com/apache/airflow/issues/39607
### Apache Airflow version 2.9.1 ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? There's been 2 recent changes to the AWS `BatchOperator`: - https://github.com/apache/airflow/pull/35789 - https://github.com/apache/airflow/pull/35808 They've added the option to specify the retry strategy for the job. Unfortunately for someone who relies on the default retry strategy, which can be set/customized in the AWS Batch job definition, this value is now ignored ### What you think should happen instead? When the user does not specify a retry_strategy in constructor `BatchOperator` it should default to None. Then it will automatically fall back to the job definition retry strategy. It should NOT be set to `{"attempt": 1}` as it is currently the case. ### How to reproduce - Create an aws job definition with a retry strategy, for example: ``` "retryStrategy": { "attempts": 5, "evaluateOnExit": [ { "onStatusReason": "Essential container in task exited*", "action": "exit" }, { "onStatusReason": "*", "action": "retry" } ] } ``` - submit an aws `BatchOperator` with no `retry_strategy` or `retry_strategy=None` - The retry strategy has been overwritten to this: ``` "retryStrategy": { "attempts": 1, "evaluateOnExit": [] }, ``` ### Operating System MWAA ### Versions of Apache Airflow Providers Using https://raw.githubusercontent.com/apache/airflow/constraints-2.8.1/constraints-3.11.txt So: - apache-airflow-providers-amazon==8.16.0 - airflow=2.8.1 But it is still happening with the latest version as far as I can tell: https://github.com/apache/airflow/blob/05b6b741cbb401f94733be46407217355741f850/airflow/providers/amazon/aws/operators/batch.py#L209 ### Deployment Official Apache Airflow Helm Chart ### Deployment details MWAA ### Anything else? Here's a quick workaround that should work even with older version of the `BatchOperator` that don't have `retry_strategy`: ```python if hasattr(operator, "retry_strategy"): operator.retry_strategy = None ``` ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
