sankalpsthakur opened a new pull request, #72397: URL: https://github.com/apache/airflow/pull/72397
The Glue task-UUID scan fallback used on retry walked `GetJobRuns` with no page cap or age cutoff. A no-match retry — the common case, when the prior attempt died before `StartJobRun` — scanned the job's entire run history. AWS lookup failures were also swallowed by `except Exception` and logged only at warning, so a still-alive run could be missed and a fresh submit would hit `ConcurrentRunsExceededException` with no error-level signal. This bounds the scan (page cap plus stop once `StartedOn` predates the Dag run), narrows reconnect lookups to `ClientError`, logs fallthrough at error with the reason, and documents the `glue:GetJobRuns` IAM requirement. Fixes https://github.com/apache/airflow/issues/71489 --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Cursor Grok 4.6 Generated-by: Cursor Grok 4.6 following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
