potiuk opened a new pull request, #66170: URL: https://github.com/apache/airflow/pull/66170
Backport of #65490 to `v3-2-test`. The fix landed on `main` on April 19 but was never backported. It was needed in the wild today: the `v3-2-test` K8s system-test job [K8S System:CeleryExecutor-3.10-v1.30.13-false](https://github.com/apache/airflow/actions/runs/25169901734/job/73801589626) hit `test_pod_affinity[backcompat]` failing with exactly this class of error — Docker Hub returned a 502 Bad Gateway pulling `docker.io/library/ubuntu:latest`, and `detect_pod_terminate_early_issues()` on `v3-2-test` doesn't yet recognise `bad gateway` as transient, so it aborted the pod before kubelet's next retry. Original commit message: > When a pod is starting, kubelet automatically retries failed image pulls > with exponential backoff. `KubernetesPodOperator` monitors the pod via > `detect_pod_terminate_early_issues()` and aborts early on what it deems > a fatal pull error — but the transient-error pattern list did not cover > registry 5xx / gateway outages. A short Docker Hub outage (502/503/504 > from `auth.docker.io` or the registry) was being classified as fatal > and would abort the pod before kubelet's next retry, even though the > pull would have succeeded once upstream recovered. > > Add \`bad gateway\`, \`service unavailable\`, \`gateway timeout\` to > \`TRANSIENT_ERROR_PATTERNS\` so these conditions let kubelet keep > retrying within the caller's \`startup_timeout\` budget. Cherry-pick conflict was a single trivial import (`_parse_log_level` doesn't exist on `v3-2-test`), resolved by keeping only the `detect_pod_terminate_early_issues` import that the new tests need. All 11 affected unit tests pass locally, including the `[ErrImagePull-rpc error: 502 Bad Gateway from auth.docker.io]` case that mirrors the failing CI run. (cherry picked from commit 938ccf3b2fc3238d36eca621a4580fd9424d161d) --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Claude Code (Opus 4.7) Generated-by: Claude Code (Opus 4.7) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
