[ https://issues.apache.org/jira/browse/SPARK-16541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved SPARK-16541. ------------------------------- Resolution: Cannot Reproduce > SparkTC application could not shutdown successfully > ---------------------------------------------------- > > Key: SPARK-16541 > URL: https://issues.apache.org/jira/browse/SPARK-16541 > Project: Spark > Issue Type: Bug > Affects Versions: 2.0.0 > Reporter: Yesha Vora > > SparkTC application in yarn-client mode was stuck at 10% progress. > {code} spark-submit --class org.apache.spark.examples.SparkTC --master > yarn-client spark-examples-assembly_*.jar {code} > It seems like SparkTC application tasks finished and printed "TC has 6254 > edges.". after that while shutting down, spark application kept getting > "ERROR LiveListenerBus: SparkListenerBus has already stopped! Dropping event > SparkListenerExecutorMetricsUpdate" > {code} > 16/07/13 08:43:37 INFO DAGScheduler: ResultStage 283 (count at > SparkTC.scala:71) finished in 42.357 s > 16/07/13 08:43:37 INFO DAGScheduler: Job 13 finished: count at > SparkTC.scala:71, took 43.137408 s > TC has 6254 edges. > 16/07/13 08:43:37 INFO ServerConnector: Stopped > ServerConnector@5e0054a2{HTTP/1.1}{0.0.0.0:4040} > 16/07/13 08:43:37 INFO ContextHandler: Stopped > o.s.j.s.ServletContextHandler@7350a22{/stages/stage/kill,null,UNAVAILABLE} > 16/07/13 08:43:37 INFO ContextHandler: Stopped > o.s.j.s.ServletContextHandler@54d56a49{/api,null,UNAVAILABLE} > 16/07/13 08:43:37 INFO ContextHandler: Stopped > o.s.j.s.ServletContextHandler@2fd52e57{/,null,UNAVAILABLE} > 16/07/13 08:43:37 INFO ContextHandler: Stopped > o.s.j.s.ServletContextHandler@7737ff3{/static,null,UNAVAILABLE} > 16/07/13 08:43:37 INFO ContextHandler: Stopped > o.s.j.s.ServletContextHandler@499d9067{/executors/threadDump/json,null,UNAVAILABLE} > 16/07/13 08:43:37 INFO ContextHandler: Stopped > o.s.j.s.ServletContextHandler@40d0c2af{/executors/threadDump,null,UNAVAILABLE} > 16/07/13 08:43:37 INFO ContextHandler: Stopped > o.s.j.s.ServletContextHandler@44ce4013{/executors/json,null,UNAVAILABLE} > 16/07/13 08:43:37 INFO ContextHandler: Stopped > o.s.j.s.ServletContextHandler@59c9a28a{/executors,null,UNAVAILABLE} > 16/07/13 08:43:37 INFO ContextHandler: Stopped > o.s.j.s.ServletContextHandler@2e784443{/environment/json,null,UNAVAILABLE} > 16/07/13 08:43:37 INFO ContextHandler: Stopped > o.s.j.s.ServletContextHandler@10240ba4{/environment,null,UNAVAILABLE} > 16/07/13 08:43:37 INFO ContextHandler: Stopped > o.s.j.s.ServletContextHandler@4ee2dd22{/storage/rdd/json,null,UNAVAILABLE} > 16/07/13 08:43:37 INFO ContextHandler: Stopped > o.s.j.s.ServletContextHandler@79ab14cd{/storage/rdd,null,UNAVAILABLE} > 16/07/13 08:43:37 INFO ContextHandler: Stopped > o.s.j.s.ServletContextHandler@731d1285{/storage/json,null,UNAVAILABLE} > 16/07/13 08:43:37 INFO ContextHandler: Stopped > o.s.j.s.ServletContextHandler@72e46ea8{/storage,null,UNAVAILABLE} > 16/07/13 08:43:37 INFO ContextHandler: Stopped > o.s.j.s.ServletContextHandler@266dcdd5{/stages/pool/json,null,UNAVAILABLE} > 16/07/13 08:43:37 INFO ContextHandler: Stopped > o.s.j.s.ServletContextHandler@17ee6dd9{/stages/pool,null,UNAVAILABLE} > 16/07/13 08:43:37 INFO ContextHandler: Stopped > o.s.j.s.ServletContextHandler@717867ea{/stages/stage/json,null,UNAVAILABLE} > 16/07/13 08:43:37 INFO ContextHandler: Stopped > o.s.j.s.ServletContextHandler@74aaadcc{/stages/stage,null,UNAVAILABLE} > 16/07/13 08:43:37 INFO ContextHandler: Stopped > o.s.j.s.ServletContextHandler@14f35a42{/stages/json,null,UNAVAILABLE} > 16/07/13 08:43:37 INFO ContextHandler: Stopped > o.s.j.s.ServletContextHandler@27ec74f8{/stages,null,UNAVAILABLE} > 16/07/13 08:43:37 INFO ContextHandler: Stopped > o.s.j.s.ServletContextHandler@148ad9f9{/jobs/job/json,null,UNAVAILABLE} > 16/07/13 08:43:37 INFO ContextHandler: Stopped > o.s.j.s.ServletContextHandler@14445e4c{/jobs/job,null,UNAVAILABLE} > 16/07/13 08:43:37 INFO ContextHandler: Stopped > o.s.j.s.ServletContextHandler@6d1557ff{/jobs/json,null,UNAVAILABLE} > 16/07/13 08:43:37 INFO ContextHandler: Stopped > o.s.j.s.ServletContextHandler@aca62b1{/jobs,null,UNAVAILABLE} > 16/07/13 08:43:37 INFO SparkUI: Stopped Spark web UI at http://xx.xx.xx:4040 > 16/07/13 08:43:37 ERROR LiveListenerBus: SparkListenerBus has already > stopped! Dropping event SparkListenerExecutorMetricsUpdate(2,WrappedArray()) > 16/07/13 08:43:37 ERROR LiveListenerBus: SparkListenerBus has already > stopped! Dropping event SparkListenerExecutorMetricsUpdate(1,WrappedArray()) > 16/07/13 08:43:47 ERROR LiveListenerBus: SparkListenerBus has already > stopped! Dropping event SparkListenerExecutorMetricsUpdate(2,WrappedArray()) > 16/07/13 08:43:47 ERROR LiveListenerBus: SparkListenerBus has already > stopped! Dropping event SparkListenerExecutorMetricsUpdate(1,WrappedArray()) > 16/07/13 08:43:57 ERROR LiveListenerBus: SparkListenerBus has already > stopped! Dropping event SparkListenerExecutorMetricsUpdate(2,WrappedArray()) > 16/07/13 08:43:57 ERROR LiveListenerBus: SparkListenerBus has already > stopped! Dropping event SparkListenerExecutorMetricsUpdate(1,WrappedArray()) > 16/07/13 08:44:07 ERROR LiveListenerBus: SparkListenerBus has already > stopped! Dropping event SparkListenerExecutorMetricsUpdate(2,WrappedArray()) > 16/07/13 08:44:07 ERROR LiveListenerBus: SparkListenerBus has already > stopped! Dropping event SparkListenerExecutorMetricsUpdate(1,WrappedArray()) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org