ferruzzi commented on code in PR #74561:
URL: https://github.com/apache/airflow/pull/74561#discussion_r4298330725
##########
providers/amazon/src/airflow/providers/amazon/aws/hooks/appflow.py:
##########
@@ -68,7 +68,7 @@ def run_flow(
waiter=self.get_waiter("run_complete", {"EXECUTION_ID":
execution_id}),
waiter_delay=poll_interval,
waiter_max_attempts=max_attempts,
- args={"flowName": flow_name},
+ args={"flowName": flow_name, "maxResults": 100},
Review Comment:
Just pointing out in case this wasn't your intent: that the waiter doesn't
paginate, so this is just one single larger page. It's not a bad change, I'm
just making sure it's the change you thought it was.
##########
providers/amazon/src/airflow/providers/amazon/aws/hooks/appflow.py:
##########
@@ -80,11 +80,21 @@ def run_flow(
return execution_id
- def _log_execution_description(self, flow_name: str, execution_id: str):
- response_desc =
self.conn.describe_flow_execution_records(flowName=flow_name)
- last_execs = {fe["executionId"]: fe for fe in
response_desc["flowExecutions"]}
- exec_details = last_execs[execution_id]
- self.log.info("Run complete, execution details: %s", exec_details)
+ def _log_execution_description(self, flow_name: str, execution_id: str) ->
None:
+ args: dict[str, Any] = {"flowName": flow_name, "maxResults": 100}
+ while True:
Review Comment:
Maybe just a style nit, but an unbounded loop seems like an unnecessary
risk, would you consider adding an escape hatch to your break condition below
like `or next_token == args.get("nextToken")` so it exits when the service
sends back the same value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
