[ https://issues.apache.org/jira/browse/SPARK-13580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172786#comment-15172786 ]
Shixiong Zhu commented on SPARK-13580: -------------------------------------- It hapens in an Akka thread. You can add {{-XX:OnOutOfMemoryError='kill -9 %p'}} to the executor java options to force the executor exit for OOM Or just upgrade to 1.6.0 which doesn't use Akka any more. > Driver makes no progress after failed to remove broadcast on Executor > --------------------------------------------------------------------- > > Key: SPARK-13580 > URL: https://issues.apache.org/jira/browse/SPARK-13580 > Project: Spark > Issue Type: Bug > Components: Streaming > Affects Versions: 1.5.2 > Reporter: Liyin Tang > Attachments: driver_jstack.txt, driver_log.txt, executor_jstack, > stderrfiltered.txt.gz > > > From Driver's log: it failed to remove broadcast data due to RPC timeout > exception from executor #11. And it also failed to get thread dump from > executor #11 due to akka.actor.ActorNotFound exception. > After that, driver waited for executor #11 to finish one task for that job. > All the other tasks are finished for that job. > However, from the executor#11's log, it didn't get that task (it got 9 other > tasks and finished them) > Since then, there is no progress in the streaming job. > I have attached the driver's log and jstack, executor's jstack. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org