Hi Dave, Could you paste the full stacktrace ? You can find it in the spark interpreter log file which is located in ZEPPELIN_HOME/logs
Xun Liu <neliu...@163.com> 于2019年3月15日周五 上午8:21写道： > Hi > > You can first execute a simple statement in spark, through sparksql, to > see if it can run normally in YARN. > If sparksql is running without problems, check the zeppelin and spark on > yarn issues. > > Also, what do you use for zeppelin-0.7.4? zeppelin-0.8.2? Is it a branch > that you maintain yourself? > > 在 2019年3月15日，上午6:31，Dave Boyd <db...@incadencecorp.com> 写道： > > All: > > I have some code that worked fine in Zeppelin 0.7.4 but I am having > issues in 0.8.2 when going from spark master of local to yarn-client. Yarn > client worked in 0.7.4. > > When my master is set to local[*] it runs just fine. However, as soon as > I switch to yarn-client I get the Cannot call methods on a stopped > SparkContext error. In looking at my yarn logs everything creates fine > and the job finishes without an error. The executors start just fine > from what I get out of yarn logs. > > Any suggestions on where to look? This happens with any note that trys > to run spark. > > If I try this very simple code: > > // Spark Version > spark.version > > I get this error: > > java.lang.IllegalStateException: Spark context stopped while waiting for > backend at > org.apache.spark.scheduler.TaskSchedulerImpl.waitBackendReady(TaskSchedulerImpl.scala:614) > at > org.apache.spark.scheduler.TaskSchedulerImpl.postStartHook(TaskSchedulerImpl.scala:169) > at org.apache.spark.SparkContext.<init>(SparkContext.scala:567) at > org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2313) at > org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:868) > at > org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:860) > at scala.Option.getOrElse(Option.scala:121) at > org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:860) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > org.apache.zeppelin.spark.BaseSparkScalaInterpreter.spark2CreateContext(BaseSparkScalaInterpreter.scala:259) > at > org.apache.zeppelin.spark.BaseSparkScalaInterpreter.createSparkContext(BaseSparkScalaInterpreter.scala:178) > at > org.apache.zeppelin.spark.SparkScala211Interpreter.open(SparkScala211Interpreter.scala:89) > at > org.apache.zeppelin.spark.NewSparkInterpreter.open(NewSparkInterpreter.java:102) > at > org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:62) > at > org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69) > at > org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:616) > at org.apache.zeppelin.scheduler.Job.run(Job.java:188) at > org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:140) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > > What am I missing? > > -- > ========= mailto:db...@incadencecorp.com <db...@incadencecorp.com> > ============ > David W. Boyd > VP, Data Solutions > 10432 Balls Ford, Suite 240 > Manassas, VA 20109 > office: +1-703-552-2862 > cell: +1-703-402-7908 > ============== http://www.incadencecorp.com/ ============ > ISO/IEC JTC1 WG9, editor ISO/IEC 20547 Big Data Reference Architecture > Chair ANSI/INCITS TC Big Data > Co-chair NIST Big Data Public Working Group Reference Architecture > First Robotic Mentor - FRC, FTC - www.iliterobotics.org > Board Member- USSTEM Foundation - www.usstem.org > > The information contained in this message may be privileged > and/or confidential and protected from disclosure. > If the reader of this message is not the intended recipient > or an employee or agent responsible for delivering this message > to the intended recipient, you are hereby notified that any > dissemination, distribution or copying of this communication > is strictly prohibited. If you have received this communication > in error, please notify the sender immediately by replying to > this message and deleting the material from any computer. > > > > > -- Best Regards Jeff Zhang
