GitHub user arkadiuszbach added a comment to the discussion: Unexpected SIGTERM 
on Tasks (Airflow 2.10.5 on GKE with KubernetesExecutor and No Resource 
Constraints)

>From the [GKE 
>docs](https://docs.cloud.google.com/kubernetes-engine/docs/best-practices/upgrading-clusters#graceful-termination),
> with the default surge-upgrade strategy, a pod has 1 hour to terminate 
>gracefully during node upgrades — after that it gets evicted, so maybe that is 
>the reason?

The cluster autoscaler could also evict the pod, but if safe-to-evict is false 
(that is the default in the helm chart) then the autoscaler won't remove the 
node at all. However, if safe-to-evict is not false, the autoscaler has its own 
graceful termination timeout that overrides the pod's 
terminationGracePeriodSeconds — and on GKE [this is not 
configurable](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/cluster-autoscaler):
 it's 10 minutes for GKE versions before 1.32.7, and 1 hour for GKE 1.32.7+. So 
even with a large terminationGracePeriodSeconds, the autoscaler will 
force-terminate the pod after that timeout.


GitHub link: 
https://github.com/apache/airflow/discussions/62978#discussioncomment-17155595

----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]

Reply via email to