jscheffl commented on code in PR #50192:
URL: https://github.com/apache/airflow/pull/50192#discussion_r2210981824
##########
providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/operators/pod.py:
##########
@@ -602,12 +603,20 @@ def get_or_create_pod(self, pod_request_obj: k8s.V1Pod,
context: Context) -> k8s
def await_pod_start(self, pod: k8s.V1Pod) -> None:
try:
- self.pod_manager.await_pod_start(
- pod=pod,
- schedule_timeout=self.schedule_timeout_seconds,
- startup_timeout=self.startup_timeout_seconds,
- check_interval=self.startup_check_interval_seconds,
+ loop = asyncio.get_event_loop()
Review Comment:
I am not sure what in parallel execution in a sync task is then the right
way. For us it is working like this since a year in production. But maybe our
environment is not representative?
Challenge is to parse and follow logs and events in parallel. There is no
API in K8s delivering both concurrently and flipping back-and forth is very in
efficient if you want to listen to log stream. Therefore we took the async
approach.
Do you have more details on where and how it is "breaking"? Which
environment?
Note that we also initially attempted to run another thread and not using
asyncio but this also was blocked by Celery and is probably also not advised.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
