[ https://issues.apache.org/jira/browse/SPARK-11104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-11104: ------------------------------------ Assignee: (was: Apache Spark) > A potential deadlock in StreamingContext.stop and stopOnShutdown > ---------------------------------------------------------------- > > Key: SPARK-11104 > URL: https://issues.apache.org/jira/browse/SPARK-11104 > Project: Spark > Issue Type: Bug > Components: Streaming > Reporter: Shixiong Zhu > > When the shutdown hook of StreamingContext and StreamingContext.stop are > running at the same time (e.g., press CTRL-C when StreamingContext.stop is > running), the following deadlock may happen: > {code} > Java stack information for the threads listed above: > =================================================== > "Thread-2": > at > org.apache.spark.streaming.StreamingContext.stop(StreamingContext.scala:699) > - waiting to lock <0x00000005405a1680> (a > org.apache.spark.streaming.StreamingContext) > at > org.apache.spark.streaming.StreamingContext.org$apache$spark$streaming$StreamingContext$$stopOnShutdown(StreamingContext.scala:729) > at > org.apache.spark.streaming.StreamingContext$$anonfun$start$1.apply$mcV$sp(StreamingContext.scala:625) > at > org.apache.spark.util.SparkShutdownHook.run(ShutdownHookManager.scala:266) > at > org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ShutdownHookManager.scala:236) > at > org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1$$anonfun$apply$mcV$sp$1.apply(ShutdownHookManager.scala:236) > at > org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1$$anonfun$apply$mcV$sp$1.apply(ShutdownHookManager.scala:236) > at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1697) > at > org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1.apply$mcV$sp(ShutdownHookManager.scala:236) > at > org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1.apply(ShutdownHookManager.scala:236) > at > org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1.apply(ShutdownHookManager.scala:236) > at scala.util.Try$.apply(Try.scala:161) > at > org.apache.spark.util.SparkShutdownHookManager.runAll(ShutdownHookManager.scala:236) > - locked <0x00000005405b6a00> (a > org.apache.spark.util.SparkShutdownHookManager) > at > org.apache.spark.util.SparkShutdownHookManager$$anon$2.run(ShutdownHookManager.scala:216) > at > org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54) > "main": > at > org.apache.spark.util.SparkShutdownHookManager.remove(ShutdownHookManager.scala:248) > - waiting to lock <0x00000005405b6a00> (a > org.apache.spark.util.SparkShutdownHookManager) > at > org.apache.spark.util.ShutdownHookManager$.removeShutdownHook(ShutdownHookManager.scala:199) > at > org.apache.spark.streaming.StreamingContext.stop(StreamingContext.scala:712) > - locked <0x00000005405a1680> (a > org.apache.spark.streaming.StreamingContext) > at > org.apache.spark.streaming.StreamingContext.stop(StreamingContext.scala:684) > - locked <0x00000005405a1680> (a > org.apache.spark.streaming.StreamingContext) > at > org.apache.spark.streaming.SessionByKeyBenchmark$.main(SessionByKeyBenchmark.scala:108) > at > org.apache.spark.streaming.SessionByKeyBenchmark.main(SessionByKeyBenchmark.scala) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:497) > at > org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:680) > at > org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180) > at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:120) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org