gbloisi-openaire commented on issue #51005:
URL: https://github.com/apache/airflow/issues/51005#issuecomment-3096077146
I got to the same issue with SparkKubernetesOperator noticing
sparkapplication are not deleted when a DAG or task is stopped by the "Mark as
Failed" action in UI.
I can confirm this is still standing as of Airflow 3.0.3 and it seems to me
a quite severe issue.
It is not working also when there is an explicit kill in the logs:
`[2025-07-21, 10:11:14] INFO - The Pod has an Event: Started container
spark-kubernetes-driver from spec.containers{spark-kubernetes-driver}:
source="airflow.providers.cncf.kubernetes.utils.pod_manager.PodManager"
Waiting until 120s to get the POD scheduled...:
source="airflow.providers.cncf.kubernetes.utils.pod_manager.PodManager"
[2025-07-21, 10:11:19] ERROR - Server indicated the task shouldn't be
running anymore. Terminating process:
detail={"detail":{"reason":"not_running","message":"TI is no longer in the
running state and task should terminate","current_state":"failed"}}:
source="task"
[2025-07-21, 10:11:19] ERROR - Task killed!: source="task"`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
