I am facing a similar issue while using Spark R. My environment:
- Spark 2.0.2 - Zeppelin: 0.6.2 - Java 1.8.0_111 - R: 3.3.1 SPARK_HOME is set. I am trying to run a simple paragraph. %r print("hello ...") I get the following exception *Interpreter Log* > # getZeppelinR > .zeppelinR = SparkR:::callJStatic("org.apache.zeppelin.spark.ZeppelinR", "getZeppelinR", hashCode) at org.apache.zeppelin.spark.ZeppelinR.waitForRScriptInitialized( ZeppelinR.java:295) at org.apache.zeppelin.spark.ZeppelinR.request(ZeppelinR.java:235) at org.apache.zeppelin.spark.ZeppelinR.eval(ZeppelinR.java:183) at org.apache.zeppelin.spark.ZeppelinR.open(ZeppelinR.java:172) at org.apache.zeppelin.spark.SparkRInterpreter.open( SparkRInterpreter.java:85) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open( LazyOpenInterpreter.java:69) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgress( LazyOpenInterpreter.java:110) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer. getProgress(RemoteInterpreterServer.java:404) at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$ Processor$getProgress.getResult(RemoteInterpreterService.java:1509) at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$ Processor$getProgress.getResult(RemoteInterpreterService.java:1494) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run( TThreadPoolServer.java:285) at java.util.concurrent.ThreadPoolExecutor.runWorker( ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run( ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) *Zeppelin Log* ERROR [2016-11-16 14:42:05,664] ({Thread-377} JobProgressPoller.java[run]:54) - Can not get or update progress org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.transport.TTransportException at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getProgress( RemoteInterpreter.java:373) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgress( LazyOpenInterpreter.java:111) at org.apache.zeppelin.notebook.Paragraph.progress(Paragraph.java:237) at org.apache.zeppelin.scheduler.JobProgressPoller.run( JobProgressPoller.java:51) Caused by: org.apache.thrift.transport.TTransportException at org.apache.thrift.transport.TIOStreamTransport.read( TIOStreamTransport.java:132) at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) at org.apache.thrift.protocol.TBinaryProtocol.readAll( TBinaryProtocol.java:429) at org.apache.thrift.protocol.TBinaryProtocol.readI32( TBinaryProtocol.java:318) at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin( TBinaryProtocol.java:219) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69) at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$ Client.recv_getProgress(RemoteInterpreterService.java:296) at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$ Client.getProgress(RemoteInterpreterService.java:281) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getProgress( RemoteInterpreter.java:370) ... 3 more When I tried run R commands using sparkR by launching $SPARK_HOME/bin/sparkR, it works. Would much appreciate any help. Thanks! Abul On Fri, Sep 16, 2016 at 5:36 PM, Hyung Sung Shim <hss...@nflabs.com> wrote: > Hello. > I think that `zeppelin.interpreter.host` and `zeppelin.interpreter.port` > values on Interpreter setting UI should be removed because Interpreter > will automatically be launched when you run a paragraph. > > 2016-09-16 3:56 GMT+09:00 Arpan Saha <arpan.s...@mindtree.com>: > >> Yes. It is a zeppelin log. Not spark interpreter log. Can you please tell >> me where can I get spark interpreter log? I don't find anything under >> zeppelin/logs/ path. >> >> Thanks, >> Arpan >> ________________________________________ >> From: DuyHai Doan [doanduy...@gmail.com] >> Sent: Thursday, September 15, 2016 11:20 AM >> To: users@zeppelin.apache.org >> Subject: Re: FW: Issue with Zeppelin setup on Datastax-Spark >> >> Is it Zeppelin log or Spark interpreter log ? I guess we need also the >> spark interpreter log to understand what is breaking Spark under the hood. >> org.apache.thrift.transport.TTransportException is just technical >> exception on inter-jvm communication >> >> On Thu, Sep 15, 2016 at 7:13 PM, Arpan Saha <arpan.s...@mindtree.com >> <mailto:arpan.s...@mindtree.com>> wrote: >> >> Hi, >> >> Please find below full stack trace. >> >> ERROR [2016-09-15 16:49:15,117] ({pool-1-thread-2} >> RemoteInterpreter.java[init]:182) - Failed to create interpreter: >> org.apache.zeppelin.spark.SparkInterpreter >> ERROR [2016-09-15 16:49:15,118] ({pool-1-thread-2} Job.java[run]:189) - >> Job failed >> org.apache.zeppelin.interpreter.InterpreterException: >> org.apache.thrift.transport.TTransportException >> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.ini >> t(RemoteInterpreter.java:183) >> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.get >> FormType(RemoteInterpreter.java:328) >> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormT >> ype(LazyOpenInterpreter.java:105) >> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java >> :260) >> at org.apache.zeppelin.scheduler.Job.run(Job.java:176) >> at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run( >> RemoteScheduler.java:328) >> at java.util.concurrent.Executors$RunnableAdapter.call(Executor >> s.java:511) >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFu >> tureTask.access$201(ScheduledThreadPoolExecutor.java:180) >> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFu >> tureTask.run(ScheduledThreadPoolExecutor.java:293) >> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool >> Executor.java:1142) >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo >> lExecutor.java:617) >> at java.lang.Thread.run(Thread.java:745) >> Caused by: org.apache.thrift.transport.TTransportException >> at org.apache.thrift.transport.TIOStreamTransport.read(TIOStrea >> mTransport.java:132) >> at org.apache.thrift.transport.TTransport.readAll(TTransport.ja >> va:86) >> at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryPr >> otocol.java:429) >> at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryPr >> otocol.java:318) >> at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin( >> TBinaryProtocol.java:219) >> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient. >> java:69) >> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ >> ice$Client.recv_createInterpreter(RemoteInterpreterService.java:184) >> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ >> ice$Client.createInterpreter(RemoteInterpreterService.java:168) >> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.ini >> t(RemoteInterpreter.java:172) >> ... 12 more >> ERROR [2016-09-15 16:49:15,118] ({Thread-18} >> RemoteInterpreterEventPoller.java[run]:88) - Can't get >> RemoteInterpreterEvent >> >> Thanks, >> Arpan. >> >> ________________________________________ >> From: DuyHai Doan [doanduy...@gmail.com<mailto:doanduy...@gmail.com>] >> Sent: Thursday, September 15, 2016 4:29 AM >> To: users@zeppelin.apache.org<mailto:users@zeppelin.apache.org> >> Subject: Re: Issue with Zeppelin setup on Datastax-Spark >> >> Can you provide the complete stack trace ? >> >> Also, worth reading my blog post about integrating Spark/Zeppelin/C* or >> DSE : http://www.doanduyhai.com/blog/?p=2325 >> >> On Thu, Sep 15, 2016 at 5:17 AM, Arpan Saha <arpan.s...@mindtree.com >> <mailto:arpan.s...@mindtree.com><mailto:arpan.s...@mindtree.com<mailto: >> arpan.s...@mindtree.com>>> wrote: >> Hi Team, >> >> I am trying to integrate Zeppelin 0.6.0 with DataStax 4.8.8 (which has >> Spark 1.4.2). After I configured following properties in zeppelin-env.sh >> when I start zeppelin daemon it started and in the browser I can see >> zeppelin is running but when I am trying to execute spark query in the >> notebook it is throwing below Error. Could you please help me to solve this >> issue. >> >> export JAVA_HOME=<JAVA Home Path> >> export SPARK_HOME=/etc/dse/spark >> export HADOOP_CONF_DIR=/etc/dse/hadoop >> export MASTER=spark://<masterhost>:7077 >> >> Also, added/updated below properties in the Spark Interpreter screen of >> Zeppelin UI. >> >> Master >> spark.app.name<http://spark.app.name><http://spark.app.name> >> spark.cassandra.auth.password >> spark.cassandra.auth.username >> spark.cassandra.connection.host >> spark.cores.max >> spark.executor.memory >> zeppelin.interpreter.host >> zeppelin.interpreter.port >> >> Trying to execute below statement in the notebook: >> %spark >> sc.version >> >> Below Error is getting: >> java.net.SocketException: Connection reset at >> java.net.SocketInputStream.read(SocketInputStream.java:209) at >> java.net.SocketInputStream.read(SocketInputStream.java:141) at >> java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at >> java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at >> java.io.BufferedInputStream.read(BufferedInputStream.java:345) at >> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127) >> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) at >> org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429) >> at >> org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318) >> at >> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219) >> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69) >> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ >> ice$Client.recv_createInterpreter(RemoteInterpreterService.java:184) at >> org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ >> ice$Client.createInterpreter(RemoteInterpreterService.java:168) at >> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:172) >> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.get >> FormType(RemoteInterpreter.java:328) at org.apache.zeppelin.interprete >> r.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:105) >> >> users-subscr...@zeppelin.apache.org<mailto:users-subscribe@z >> eppelin.apache.org><mailto:users-subscr...@zeppelin.apache.org<mailto: >> users-subscr...@zeppelin.apache.org>> >> >> Thanks, >> Arpan. >> >> ________________________________ >> Notice: This e-mail message has been sent by an employee or contractor of >> American Express Global Business Travel (“GBT”), a joint venture that is >> not wholly-owned by American Express Company or any of its subsidiaries >> (“American Express”). E-mail from GBT employees and contractors may >> continue to utilize an American Express domain for some period; however >> American Express is not responsible for the content of this email, which is >> controlled by GBT. “American Express Global Business Travel”, “American >> Express” and the American Express Logo are trademarks of American Express >> and are used by GBT under limited license. >> ________________________________ >> Important : Cet email a été envoyé par un employé ou un sous-traitant >> d’American Express Global Business Travel (« GBT »), une joint-venture qui >> n’est pas intégralement détenue par American Express Company ou l’une de >> ses filiales (« American Express »). Les emails des employés et >> sous-traitants de GBT peuvent continuer de comporter un nom de domaine >> d’American Express pour une certaine période ; toutefois, American Express >> n’est pas responsable du contenu de cet email, qui est contrôlé par GBT. « >> American Express Global Business Travel », « American Express » et le logo >> American Express sont des marques d’American Express et sont utilisées par >> GBT aux termes d’une licence limitée. >> >> ________________________________ >> >> http://www.mindtree.com/email/disclaimer.html >> >> >> >