seanmuth commented on code in PR #69058:
URL: https://github.com/apache/airflow/pull/69058#discussion_r3546619738
##########
providers/cncf/kubernetes/provider.yaml:
##########
@@ -468,6 +468,40 @@ config:
type: integer
example: ~
default: "0"
+ pod_launch_failure_retries:
+ description: |
+ The number of times the executor will transparently requeue a task
whose worker pod
+ failed before the task process started running (for example a node
drain, autoscaler
+ scale-down, node boot race, or transient image pull failure). The
task instance is
+ still in ``queued`` state in these cases, meaning no task code ran,
so requeuing does
+ not consume a task-level retry.
+
+ This changes the previous default behavior: such tasks are now
requeued once before
+ failing instead of failing on the first pod failure. Set this to 0
to restore the
+ previous behavior (fail immediately, no requeue).
+
+ Use -1 for unlimited requeues, but with caution: a pod that fails on
every launch
+ (for example a misconfigured image that can never be pulled) will be
requeued forever,
+ and with the default ``delete_worker_pods_on_failure = False`` the
failed pods are not
+ cleaned up, so they accumulate.
+ version_added: 10.19.0
Review Comment:
Done in `cb32253819` — bumped both new configs to `version_added: 10.20.0`
and regenerated `get_provider_info.py` to match. Thanks for the correction!
---
Drafted-by: Claude Code (Opus 4.8); reviewed by @seanmuth before posting
##########
providers/cncf/kubernetes/provider.yaml:
##########
@@ -468,6 +468,40 @@ config:
type: integer
example: ~
default: "0"
+ pod_launch_failure_retries:
+ description: |
+ The number of times the executor will transparently requeue a task
whose worker pod
+ failed before the task process started running (for example a node
drain, autoscaler
+ scale-down, node boot race, or transient image pull failure). The
task instance is
+ still in ``queued`` state in these cases, meaning no task code ran,
so requeuing does
+ not consume a task-level retry.
+
+ This changes the previous default behavior: such tasks are now
requeued once before
+ failing instead of failing on the first pod failure. Set this to 0
to restore the
+ previous behavior (fail immediately, no requeue).
+
+ Use -1 for unlimited requeues, but with caution: a pod that fails on
every launch
+ (for example a misconfigured image that can never be pulled) will be
requeued forever,
+ and with the default ``delete_worker_pods_on_failure = False`` the
failed pods are not
+ cleaned up, so they accumulate.
+ version_added: 10.19.0
+ type: integer
+ example: ~
+ default: "1"
+ pod_launch_failure_excluded_container_reasons:
+ description: |
+ Comma-separated list of container termination reasons that are
excluded from the
+ ``pod_launch_failure_retries`` requeue path even when the task
instance is still in
+ ``queued`` state. Pods that fail with an excluded reason consume a
normal task retry
+ instead of being transparently requeued. The default ``Error``
covers the case where
+ the container started executing but the worker process exited before
writing
+ ``running`` to the database, which is most likely an
Airflow-specific startup error
+ rather than a transient infrastructure event. Set to an empty value
to requeue these
+ cases as well.
+ version_added: 10.19.0
Review Comment:
Applied here too in `cb32253819` (same commit as the other `version_added`
bump).
---
Drafted-by: Claude Code (Opus 4.8); reviewed by @seanmuth before posting
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
