Devaraj K created SPARK-8754:
--------------------------------
Summary: YarnClientSchedulerBackend doesn't stop gracefully in
failure conditions
Key: SPARK-8754
URL: https://issues.apache.org/jira/browse/SPARK-8754
Project: Spark
Issue Type: Bug
Components: YARN
Affects Versions: 1.4.0
Reporter: Devaraj K
Priority: Minor
{code:xml}
java.lang.NullPointerException
at
org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.stop(YarnClientSchedulerBackend.scala:151)
at
org.apache.spark.scheduler.TaskSchedulerImpl.stop(TaskSchedulerImpl.scala:421)
at org.apache.spark.scheduler.DAGScheduler.stop(DAGScheduler.scala:1447)
at org.apache.spark.SparkContext.stop(SparkContext.scala:1651)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:572)
at org.apache.spark.examples.SparkPi$.main(SparkPi.scala:28)
at org.apache.spark.examples.SparkPi.main(SparkPi.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:621)
at
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:170)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:193)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:112)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
{code}
If the application has FINISHED/FAILED/KILLED or failed to launch application
master, monitorThread is not getting initialized but monitorThread.interrupt()
is getting invoked as part of stop() without any check and It is causing to
throw NPE and also it is preventing to stop the client.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
