[
https://issues.apache.org/jira/browse/SPARK-9097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14640046#comment-14640046
]
KaiXinXIaoLei commented on SPARK-9097:
--------------------------------------
[~srowen], This problem, I analysis code recently. I have a questions.
In this problem, executor 52 is lost because "no recent heartbeats" at
2015-07-08 15:03:30, , so "2015-07-08 15:03:30,585 | INFO | Removed 52
successfully in removeExecutor" .
But in log:
2015-07-08 16:56:31,740 | INFO |
[sparkDriver-akka.actor.default-dispatcher-43] | No response from remote.
Handshake timed out or transport failure detector triggered.
2015-07-08 16:56:31,741 | ERROR |
[sparkDriver-akka.actor.default-dispatcher-45] | Lost executor 52 on linux-174:
remote Rpc client disassociated
2015-07-08 16:56:31,741 | WARN | [sparkDriver-akka.actor.default-dispatcher-6]
| Association with remote system [akka.tcp://sparkExecutor@linux-174:23326] has
failed, address is now gated for [5000] ms. Reason is: [Disassociated].
2015-07-08 16:56:31,741 | INFO |
[sparkDriver-akka.actor.default-dispatcher-45] | Re-queueing tasks for 52 from
TaskSet 167.0
2015-07-08 16:56:31,742 | INFO | [dag-scheduler-event-loop] | Executor lost:
52 (epoch 29)
2015-07-08 16:56:31,742 | INFO |
[sparkDriver-akka.actor.default-dispatcher-48] | Trying to remove executor 52
from BlockManagerMaster.
2015-07-08 16:56:31,742 | INFO | [dag-scheduler-event-loop] | Removed 52
successfully in removeExecutor
So i want to know why executor 52 is removed again? Is it not removed
successfully at 15:03:30? Thanks.
I think i make clear.Thanks.
> Tasks are not completed but the number of executor is zero
> ----------------------------------------------------------
>
> Key: SPARK-9097
> URL: https://issues.apache.org/jira/browse/SPARK-9097
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.4.0
> Reporter: KaiXinXIaoLei
> Attachments: number of executor is zero.png, runing tasks.png
>
>
> I set the value of "spark.dynamicAllocation.enabled" is true, and set
> "spark.dynamicAllocation.minExecutors = 0". I submit tasks to run. Tasks are
> not completed, but the number of executor is zero.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
