Dongjoon Hyun created SPARK-39324:
-------------------------------------
Summary: Log ExecutorDecommission as INFO level in
TaskSchedulerImpl
Key: SPARK-39324
URL: https://issues.apache.org/jira/browse/SPARK-39324
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 3.4.0
Reporter: Dongjoon Hyun
{code}
22/05/28 01:25:28 INFO KubernetesClusterSchedulerBackend: Decommission
executors: 8
22/05/28 01:25:28 INFO KubernetesClusterSchedulerBackend: Notify executor 8 to
decommissioning.
22/05/28 01:25:28 INFO BlockManagerMasterEndpoint: Mark BlockManagers
(BlockManagerId(8, 100.103.40.13, 43353, None)) as being decommissioning.
22/05/28 01:25:29 ERROR TaskSchedulerImpl: Lost executor 8 on 100.103.40.13:
Executor decommission.
22/05/28 01:25:29 INFO ExecutorMonitor: Executor 8 is removed. Remove reason
statistics: ...
22/05/28 01:25:29 INFO DAGScheduler: Executor lost: 8 (epoch 7)
22/05/28 01:25:29 INFO BlockManagerMasterEndpoint: Trying to remove executor 8
from BlockManagerMaster.
22/05/28 01:25:29 INFO BlockManagerMasterEndpoint: Removing block manager
BlockManagerId(8, 100.103.40.13, 43353, None)
22/05/28 01:25:29 INFO BlockManagerMaster: Removed 8 successfully in
removeExecutor
22/05/28 01:25:29 INFO DAGScheduler: Shuffle files lost for executor: 8 (epoch
7)
22/05/28 01:25:34 INFO BlockManagerMaster: Removal of executor 8 requested
22/05/28 01:25:34 INFO BlockManagerMasterEndpoint: Trying to remove executor 8
from BlockManagerMaster.
22/05/28 01:25:34 INFO
KubernetesClusterSchedulerBackend$KubernetesDriverEndpoint: Asked to remove
non-existent executor 8
{code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
