kosteev commented on code in PR #52662:
URL: https://github.com/apache/airflow/pull/52662#discussion_r2230368863
##########
providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/executors/kubernetes_executor_utils.py:
##########
@@ -530,7 +530,11 @@ def terminate(self) -> None:
self.log.debug("Terminating kube_watchers...")
for kube_watcher in self.kube_watchers.values():
kube_watcher.terminate()
- kube_watcher.join()
+ kube_watcher.join(timeout=60)
Review Comment:
I saw that in the referenced issue this is indeed happening in case of some
connection issue.
However, what I observe is with fixed number of scheduler cycles (e.g. 1000
loops) - the issue happens on every scheduler termination/stop. Maybe with 1000
loops or couple of hours running scheduler, inevitably one of the connection
will have an issue and it results in this problem.
So, scheduler availability looks like this (every 1.5 hours on scheduler
restart, it is stuck to terminate):
<img width="1900" height="228" alt="image"
src="https://github.com/user-attachments/assets/56ddcce1-2b93-4e20-9e14-d02c0ea6da30";
/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
