jingi723 opened a new pull request, #73885: URL: https://github.com/apache/airflow/pull/73885
Allow Dag authors to stop retries for known application failures reported by an ECS container. For example, with `retries=2` and `fail_on_exit_code=2`, an invalid-input exit code fails the Airflow task after its first attempt, while other exit codes retain the normal retry behavior. The option accepts an integer or a container of integers, matching the existing `skip_on_exit_code` interface. It defaults to `None` and applies to synchronous and resumed deferrable execution. Exit code zero remains successful, and a missing exit code is not inferred as a match. This completes the fail-without-retry part originally proposed in #47256; #63274 subsequently added the skip option. The decision uses the existing DescribeTasks response, avoiding exception-message parsing or an additional AWS lookup to recover the exit code. A selected code raises `AirflowFailException`, including on versions supporting generic `retry_policy`. For multiple containers, a matching nonzero code from any stopped container takes precedence over sibling container failures and skip codes, irrespective of container order. Fail also takes precedence when the same code appears in both options. Existing task-level startup, host-termination and AWS API error handling remains in place. These semantics and the no-wait limitation are documented. Validation on current main (`ac88accf4edc6ed70557d5e9554e36adc04ed131`), Linux/amd64 Breeze, Python 3.10.21: - 43 new SDK-boundary cases; 152 related ECS operator/hook/trigger tests passed in both serial and two-worker non-DB runs. - 20 `Dag.test()` cases passed through the Task Runner, Execution API routes and SQLite DB, checking final state, actual attempt counts, callbacks and ARN XCom. Deferred cases run the real trigger inline with an aiobotocore Stubber. Original default-behavior cases also passed against the unmodified operator. - The full Amazon test tree passed in five fresh-process groups: 5,280 passed, 44 skipped. The transfers group used a container-local SSH server for the existing SFTP tests. - 10 existing SDK no-retry/retry-policy tests, regular/manual prek and provider mypy passed. AWS responses use SDK Stubbers. Execution API transport is in-process and triggers run inline; live AWS, separately deployed worker/triggerer processes, and the full provider/version CI matrix were not run locally. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Codex (GPT-6) Generated-by: Codex (GPT-6) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) --- * Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged. * For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed. * When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). * For significant user-facing changes create newsfragment: `{pr_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). You can add this file in a follow-up commit after the PR is created so you know the PR number. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
