[
https://issues.apache.org/jira/browse/SPARK-30357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-30357:
---------------------------------
Labels: (was: newbie)
> SparkContext: Invoking stop() from shutdown hook
> ------------------------------------------------
>
> Key: SPARK-30357
> URL: https://issues.apache.org/jira/browse/SPARK-30357
> Project: Spark
> Issue Type: Bug
> Components: Deploy, Kubernetes, MLlib, PySpark
> Affects Versions: 2.4.4
> Environment: pivotal kubernetes container service PKS having 3master
> nodes and 3 worker nodes
>
> OS : ubuntu for k8-s cluster
>
>
> Reporter: Veera
> Priority: Major
>
> I'm getting below error while running spark-submit job in kubernetes , i
> didn't get any specific error
> {code}
> 19/12/26 07:38:18 INFO SparkContext: Invoking stop() from shutdown hook
> 19/12/26 07:38:18 INFO SparkUI: Stopped Spark web UI at
> http://spark-ml-test-1577345808987-driver-svc.spark-jobs.svc:4040
> 19/12/26 07:38:18 INFO KubernetesClusterSchedulerBackend: Shutting down all
> executors
> 19/12/26 07:38:18 INFO
> KubernetesClusterSchedulerBackend$KubernetesDriverEndpoint: Asking each
> executor to shut down
> 19/12/26 07:38:18 WARN ExecutorPodsWatchSnapshotSource: Kubernetes client has
> been closed (this is expected if the application is shutting down.)
> 19/12/26 07:38:18 INFO MapOutputTrackerMasterEndpoint:
> MapOutputTrackerMasterEndpoint stopped!
> 19/12/26 07:38:18 INFO MemoryStore: MemoryStore cleared
> 19/12/26 07:38:18 INFO BlockManager: BlockManager stopped
> 19/12/26 07:38:18 INFO BlockManagerMaster: BlockManagerMaster stopped
> 19/12/26 07:38:18 INFO
> OutputCommitCoordinator$OutputCommitCoordinatorEndpoint:
> OutputCommitCoordinator stopped!
> 19/12/26 07:38:18 INFO SparkContext: Successfully stopped SparkContext
> 19/12/26 07:38:18 INFO ShutdownHookManager: Shutdown hook called
> 19/12/26 07:38:18 INFO ShutdownHookManager: Deleting directory
> /var/data/spark-ec98f5d3-022d-4eae-87aa-c23a96c77555/spark-b757b85f-3709-4215-8208-95db840a294a/pyspark-665955c9-0dc6-49ee-9b57-0392be430f10
> 19/12/26 07:38:18 INFO ShutdownHookManager: Deleting directory
> /tmp/spark-fa047fa8-b573-44aa-ab65-e235fb0d5b09
> 19/12/26 07:38:18 INFO ShutdownHookManager: Deleting directory
> /var/data/spark-ec98f5d3-022d-4eae-87aa-c23a96c77555/spark-b757b85f-3709-4215-8208-95db840a294a
> {code}
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
