Ran Haim created SPARK-25527:
--------------------------------
Summary: Job stuck waiting for last stage to start
Key: SPARK-25527
URL: https://issues.apache.org/jira/browse/SPARK-25527
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 2.1.0
Reporter: Ran Haim
Sometimes it can somehow happen that a job is stuck waiting for the last stage
to start.
There are no Tasks waiting for completion, and the job just hangs.
There are available Executors for the job to run.
I do not know how to reproduce this, all I know is that it happens randomly
after couple days of hard load.
Another thing that might help is that it seems to happen when some tasks fail
because one or more executors killed (due to memory issues or something).
Those tasks eventually do get finished by other executors because of retries,
but the next stage hangs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
