[
https://issues.apache.org/jira/browse/YUNIKORN-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492929#comment-17492929
]
Anuraag Nalluri commented on YUNIKORN-946:
------------------------------------------
Hi [~ashutosh-pepper], I wanted to take a look at this but am not too familiar
with the k8s spark operator. I used the quickstart guide and ran the spark-pi
example on my docker-desktop with YuniKorn scheduler active. Once the executor
pods were in "Running" state, I deleted the spark-pi application. However, the
YK UI successfully transitions the application to "completed" and sets used
resources to 0.
I do not see leaked resources of running executors. Is there a specific
application/example in step 1 you ran to produce this scenario 1? If so, I'd
like to try it out – thanks.
> Accounting resources for deleted executor pods
> ----------------------------------------------
>
> Key: YUNIKORN-946
> URL: https://issues.apache.org/jira/browse/YUNIKORN-946
> Project: Apache YuniKorn
> Issue Type: Bug
> Components: core - scheduler
> Affects Versions: 0.11
> Reporter: Ashutosh Singh
> Priority: Critical
> Attachments: image-2021-11-16-23-17-42-819.png,
> image-2021-11-16-23-18-28-349.png
>
>
> Even when executors are deleted, YK UI shows that resources are consumed by
> the pod (the one which is already deleted). _kubectl get pods_ does not show
> the executor but YK UI shows the information of a deleted pod consuming
> resources even after few hours.
> It results into leaking cluster resources.
> Steps:
> # Run a spark application using k8s spark operator
> # Wait for executors to be in running state.
> # Delete the application using `kubectl delete sparkapplications <appName>`
> OR `kubectl delete {-}{{-}}f <yaml\{-}file>`
> # All the driver and executor pods would be deleted. check `kubectl get pods`
> # However, YK UI still shows some of the executors running and consuming
> resources. It leads to leak of the resource as they are considered as used
> and could not be used by pending pods.
> More details:
> [https://yunikornworkspace.slack.com/archives/CLNUW68MU/p1637126093006900]
> !image-2021-11-16-23-18-28-349.png|width=534,height=323!
>
> !image-2021-11-16-23-17-42-819.png|width=583,height=353!
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
