mo-rieger commented on issue #70585:
URL: https://github.com/apache/airflow/issues/70585#issuecomment-5438774418
I worked around this issue by creating a subclass of KubernetesJobOperator
that adds a delay to the retry loop in get_pods, e.g.:
```
POD_DISCOVERY_POLL_INTERVAL = 1
class CusomKubernetesJobOperator(KubernetesJobOperator):
"""Waits for the Job controller to create the pods. Drop once
cncf-kubernetes ships https://github.com/apache/airflow/issues/70585""";
def get_pods(self, pod_request_obj, context, *, exclude_checked=True):
label_selector = self._build_find_pod_label_selector(
context, exclude_checked=exclude_checked
)
pod_list = []
for attempt in range(self.discover_pods_retry_number + 1):
if attempt:
time.sleep(POD_DISCOVERY_POLL_INTERVAL)
pod_list = self.client.list_namespaced_pod(
namespace=pod_request_obj.metadata.namespace,
label_selector=label_selector,
).items
if len(pod_list) >= (self.parallelism or 1):
break
if not pod_list:
raise AirflowException(f"No pods running with labels
{label_selector}")
for pod_instance in pod_list:
self.log_matching_pod(pod=pod_instance, context=context)
return pod_list
```
But I would appreciate a review of the linked PR by a maintainer :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
