dongjoon-hyun commented on issue #863: URL: https://github.com/apache/spark-kubernetes-operator/issues/863#issuecomment-5790734535
Thank you for the detailed report, @robertpi. The root cause analysis is correct. `observeDriverTermination()` returned early on empty `containerStatuses` and never checked the pod phase. I made a PR to fix it. It checks the pod phase (`Failed` / `Succeeded`) before the empty-`containerStatuses` guard. - https://github.com/apache/spark-kubernetes-operator/pull/867 Two notes: 1. With the default `driverStartTimeoutMillis` (300000), such an application should have moved to `DriverStartTimedOut` after about 5 minutes. It shouldn't stay in `DriverRequested` forever. Did you set `spec.applicationTolerations.applicationTimeoutConfig.driverStartTimeoutMillis` to `0` or a very large value? If you share the `applicationTolerations` from your CR, we can confirm. 2. I didn't use the `DisruptionTarget` condition as a termination signal. Kubernetes can add it while a pod is still running, for example during preemption or graceful termination. PodGC sets the phase to `Failed` anyway, so the phase check is enough. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
