[
https://issues.apache.org/jira/browse/SPARK-37128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17435189#comment-17435189
]
JacobZheng commented on SPARK-37128:
My Spark version is 3.0.1 and I run Spark standalone on k8s.I don't know how to
reproduce it, so it doesn't always come up. Sometimes it appears when an oom
exception occurs in the executor. [~hyukjin.kwon]
> Application has been removed by master but driver still running
> ---
>
> Key: SPARK-37128
> URL: https://issues.apache.org/jira/browse/SPARK-37128
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
>Affects Versions: 3.0.1
>Reporter: JacobZheng
>Priority: Major
>
> {code:java}
> 21/08/30 10:27:31 INFO Master: Removing executor app-20210827190502-0030/1
> because it is EXITED
> 21/08/30 10:27:31 INFO Master: Launching executor app-20210827190502-0030/4
> on worker worker-20210826183405-10.39.0.69-37147
> 21/08/30 10:27:31 INFO Master: 10.39.0.68:47160 got disassociated, removing
> it.
> 21/08/30 10:27:31 INFO Master: 10.39.0.68:35160 got disassociated, removing
> it.
> 21/08/30 10:27:31 INFO Master: Removing app app-20210827190502-0030
> 21/08/30 10:27:31 WARN Master: Got status update for unknown executor
> app-20210827190502-0030/4
> 21/08/30 10:27:31 WARN Master: Got status update for unknown executor
> app-20210827190502-0030/4
> 21/08/30 10:27:46 WARN Master: Got status update for unknown executor
> app-20210827190502-0030/2
> 21/08/30 10:27:48 WARN Master: Got status update for unknown executor
> app-20210827190502-0030/0
> 21/08/30 10:27:50 WARN Master: Got status update for unknown executor
> app-20210827190502-0030/3{code}
> As the logs show, Spark master removed my application. But my driver process
> is still running. I would like to know what could be the cause of this and
> how I can avoid it.
>
> My Spark version is 3.0.1 and I run Spark standalone on k8s.I don't know how
> to reproduce it, so it doesn't always come up. Sometimes it appears when an
> oom exception occurs in the executor
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org