ashokballolli commented on code in PR #77069:
URL: https://github.com/apache/airflow/pull/77069#discussion_r4619333363
##########
providers/google/src/airflow/providers/google/cloud/hooks/dataflow.py:
##########
@@ -1602,10 +1609,17 @@ async def get_latest_job_by_name(
:return: The job, or None if no job has this name.
"""
job_name = job_name.lower()
- jobs = await self.list_jobs(project_id=project_id, location=location)
- matching_jobs = [job async for job in jobs if job.name == job_name]
- # create_time is typed as protobuf Timestamp, but proto-plus returns
it as a datetime.
- return max(matching_jobs, key=lambda job: cast("datetime",
job.create_time), default=None)
+ jobs = await self.list_jobs(project_id=project_id, location=location,
name=job_name)
+ latest_job: Job | None = None
+ async for job in jobs:
+ if job.name != job_name:
+ continue
+ # create_time is typed as protobuf Timestamp, but proto-plus
returns it as a datetime.
+ if latest_job is None or cast("datetime", job.create_time) > cast(
+ "datetime", latest_job.create_time
+ ):
+ latest_job = job
+ return latest_job
Review Comment:
Sorry, my mistake. I changed it to a loop to avoid building the list,
because `max()` cannot take the async pager directly. But it made the code
harder to read, so I changed it back to the earlier version, removed the name
check here also, and pushed the change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]