SameerMesiah97 commented on code in PR #74082:
URL: https://github.com/apache/airflow/pull/74082#discussion_r4169925986
##########
providers/amazon/src/airflow/providers/amazon/aws/operators/glue.py:
##########
@@ -347,6 +345,7 @@ def _find_job_run_id_by_task_uuid(self, task_uuid: str) ->
tuple[str, str] | Non
next_token = response.get("NextToken")
if not next_token:
return None
+ return None
Review Comment:
1. Why 100 here? Seems rather arbitrry. This is a pagination loop, so the
cap could prevent us from finding a matching run if AWS still returns a
`NextToken` after 100 pages. We already exit when there is no next token. Could
we keep `while True` and address the specific condition causing the infinite
loop instead?
2. If we keep a cap, we should raise an informative exception when it is
reached rather than silently return `None`. Otherwise, the caller cannot
distinguish between “no matching run found” and “we stopped searching before
checking all pages”, which could hide the underlying issue and make it
difficult to diagnose.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
