Re: FW: Issue with Zeppelin setup on Datastax-Spark
l.TBinaryProtocol.readI32(TBinaryPr
>>>> otocol.java:318)
>>>> at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(
>>>> TBinaryProtocol.java:219)
>>>> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
>>>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>>>> ice$Client.recv_getProgress(RemoteInterpreterService.java:296)
>>>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>>>> ice$Client.getProgress(RemoteInterpreterService.java:281)
>>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.get
>>>> Progress(RemoteInterpreter.java:370)
>>>> ... 3 more
>>>>
>>>>
>>>>
>>>> When I tried run R commands using sparkR by launching
>>>> $SPARK_HOME/bin/sparkR, it works.
>>>>
>>>>
>>>> Would much appreciate any help.
>>>>
>>>> Thanks!
>>>> Abul
>>>>
>>>>
>>>> On Fri, Sep 16, 2016 at 5:36 PM, Hyung Sung Shim
>>>> wrote:
>>>>
>>>>> Hello.
>>>>> I think that `zeppelin.interpreter.host` and `zeppelin.interpreter.port`
>>>>> values on Interpreter setting UI should be removed because Interpreter
>>>>> will automatically be launched when you run a paragraph.
>>>>>
>>>>> 2016-09-16 3:56 GMT+09:00 Arpan Saha :
>>>>>
>>>>>> Yes. It is a zeppelin log. Not spark interpreter log. Can you please
>>>>>> tell me where can I get spark interpreter log? I don't find anything
>>>>>> under
>>>>>> zeppelin/logs/ path.
>>>>>>
>>>>>> Thanks,
>>>>>> Arpan
>>>>>>
>>>>>> From: DuyHai Doan [doanduy...@gmail.com]
>>>>>> Sent: Thursday, September 15, 2016 11:20 AM
>>>>>> To: users@zeppelin.apache.org
>>>>>> Subject: Re: FW: Issue with Zeppelin setup on Datastax-Spark
>>>>>>
>>>>>> Is it Zeppelin log or Spark interpreter log ? I guess we need also
>>>>>> the spark interpreter log to understand what is breaking Spark under the
>>>>>> hood. org.apache.thrift.transport.TTransportException is just
>>>>>> technical exception on inter-jvm communication
>>>>>>
>>>>>> On Thu, Sep 15, 2016 at 7:13 PM, Arpan Saha >>>>> <mailto:arpan.s...@mindtree.com>> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Please find below full stack trace.
>>>>>>
>>>>>> ERROR [2016-09-15 16:49:15,117] ({pool-1-thread-2}
>>>>>> RemoteInterpreter.java[init]:182) - Failed to create interpreter:
>>>>>> org.apache.zeppelin.spark.SparkInterpreter
>>>>>> ERROR [2016-09-15 16:49:15,118] ({pool-1-thread-2} Job.java[run]:189)
>>>>>> - Job failed
>>>>>> org.apache.zeppelin.interpreter.InterpreterException:
>>>>>> org.apache.thrift.transport.TTransportException
>>>>>> at org.apache.zeppelin.interprete
>>>>>> r.remote.RemoteInterpreter.init(RemoteInterpreter.java:183)
>>>>>> at org.apache.zeppelin.interprete
>>>>>> r.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:328)
>>>>>> at org.apache.zeppelin.interprete
>>>>>> r.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:105)
>>>>>> at org.apache.zeppelin.notebook.P
>>>>>> aragraph.jobRun(Paragraph.java:260)
>>>>>> at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
>>>>>> at org.apache.zeppelin.scheduler.
>>>>>> RemoteScheduler$JobRunner.run(RemoteScheduler.java:328)
>>>>>> at java.util.concurrent.Executors
>>>>>> $RunnableAdapter.call(Executors.java:511)
>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>>>>> at java.util.concurrent.Scheduled
>>>>>> ThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledT
>>>>>> hreadPoolExecutor.java:180)
>>>>>> at java.util.concurrent.Scheduled
>>>>>> ThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPo
>>>>>> olExecut
Re: FW: Issue with Zeppelin setup on Datastax-Spark
ote.RemoteInterpreter.get
>>> Progress(RemoteInterpreter.java:370)
>>> ... 3 more
>>>
>>>
>>>
>>> When I tried run R commands using sparkR by launching
>>> $SPARK_HOME/bin/sparkR, it works.
>>>
>>>
>>> Would much appreciate any help.
>>>
>>> Thanks!
>>> Abul
>>>
>>>
>>> On Fri, Sep 16, 2016 at 5:36 PM, Hyung Sung Shim
>>> wrote:
>>>
>>>> Hello.
>>>> I think that `zeppelin.interpreter.host` and `zeppelin.interpreter.port`
>>>> values on Interpreter setting UI should be removed because Interpreter
>>>> will automatically be launched when you run a paragraph.
>>>>
>>>> 2016-09-16 3:56 GMT+09:00 Arpan Saha :
>>>>
>>>>> Yes. It is a zeppelin log. Not spark interpreter log. Can you please
>>>>> tell me where can I get spark interpreter log? I don't find anything under
>>>>> zeppelin/logs/ path.
>>>>>
>>>>> Thanks,
>>>>> Arpan
>>>>>
>>>>> From: DuyHai Doan [doanduy...@gmail.com]
>>>>> Sent: Thursday, September 15, 2016 11:20 AM
>>>>> To: users@zeppelin.apache.org
>>>>> Subject: Re: FW: Issue with Zeppelin setup on Datastax-Spark
>>>>>
>>>>> Is it Zeppelin log or Spark interpreter log ? I guess we need also the
>>>>> spark interpreter log to understand what is breaking Spark under the hood.
>>>>> org.apache.thrift.transport.TTransportException is just technical
>>>>> exception on inter-jvm communication
>>>>>
>>>>> On Thu, Sep 15, 2016 at 7:13 PM, Arpan Saha >>>> <mailto:arpan.s...@mindtree.com>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> Please find below full stack trace.
>>>>>
>>>>> ERROR [2016-09-15 16:49:15,117] ({pool-1-thread-2}
>>>>> RemoteInterpreter.java[init]:182) - Failed to create interpreter:
>>>>> org.apache.zeppelin.spark.SparkInterpreter
>>>>> ERROR [2016-09-15 16:49:15,118] ({pool-1-thread-2} Job.java[run]:189)
>>>>> - Job failed
>>>>> org.apache.zeppelin.interpreter.InterpreterException:
>>>>> org.apache.thrift.transport.TTransportException
>>>>> at org.apache.zeppelin.interprete
>>>>> r.remote.RemoteInterpreter.init(RemoteInterpreter.java:183)
>>>>> at org.apache.zeppelin.interprete
>>>>> r.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:328)
>>>>> at org.apache.zeppelin.interprete
>>>>> r.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:105)
>>>>> at org.apache.zeppelin.notebook.P
>>>>> aragraph.jobRun(Paragraph.java:260)
>>>>> at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
>>>>> at org.apache.zeppelin.scheduler.
>>>>> RemoteScheduler$JobRunner.run(RemoteScheduler.java:328)
>>>>> at java.util.concurrent.Executors
>>>>> $RunnableAdapter.call(Executors.java:511)
>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>>>> at java.util.concurrent.Scheduled
>>>>> ThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledT
>>>>> hreadPoolExecutor.java:180)
>>>>> at java.util.concurrent.Scheduled
>>>>> ThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPo
>>>>> olExecutor.java:293)
>>>>> at java.util.concurrent.ThreadPoo
>>>>> lExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>>>> at java.util.concurrent.ThreadPoo
>>>>> lExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>>>> at java.lang.Thread.run(Thread.java:745)
>>>>> Caused by: org.apache.thrift.transport.TTransportException
>>>>> at org.apache.thrift.transport.TI
>>>>> OStreamTransport.read(TIOStreamTransport.java:132)
>>>>> at org.apache.thrift.transport.TT
>>>>> ransport.readAll(TTransport.java:86)
>>>>> at org.apache.thrift.protocol.TBi
>>>>> naryProtocol.readAll(TBinaryProtocol.java:429)
>>>>> at org.apache.thrift.protocol.TBi
>>>>> naryProtocol.readI32(TBinaryProtocol.java:318)
>>>>> at org.apache.th
Re: FW: Issue with Zeppelin setup on Datastax-Spark
I recommend downloading my pre-built Zeppelin for Datastax. Shared folder
link: https://drive.google.com/folderview?id=0B6wR2aj4Cb6wQ01aR3ItR0xUNms
On Wed, Nov 16, 2016 at 11:13 AM, DuyHai Doan wrote:
> Ok I understand why you have issue.
>
> You are using Spark 2.0.2 and latest Datastax 5.0.3 is still using Spark
> version 1.6.X
>
>
>
> On Wed, Nov 16, 2016 at 10:23 AM, Abul Basar wrote:
>
>> I am facing a similar issue while using Spark R.
>>
>> My environment:
>>
>>- Spark 2.0.2
>>- Zeppelin: 0.6.2
>>- Java 1.8.0_111
>>- R: 3.3.1
>>
>>
>> SPARK_HOME is set. I am trying to run a simple paragraph.
>>
>> %r
>> print("hello ...")
>>
>> I get the following exception
>>
>> *Interpreter Log*
>> > # getZeppelinR
>>
>> > .zeppelinR = SparkR:::callJStatic("org.apache.zeppelin.spark.ZeppelinR",
>> "getZeppelinR", hashCode)
>>
>> at org.apache.zeppelin.spark.ZeppelinR.waitForRScriptInitialize
>> d(ZeppelinR.java:295)
>> at org.apache.zeppelin.spark.ZeppelinR.request(ZeppelinR.java:235)
>> at org.apache.zeppelin.spark.ZeppelinR.eval(ZeppelinR.java:183)
>> at org.apache.zeppelin.spark.ZeppelinR.open(ZeppelinR.java:172)
>> at org.apache.zeppelin.spark.SparkRInterpreter.open(SparkRInter
>> preter.java:85)
>> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(Laz
>> yOpenInterpreter.java:69)
>> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgr
>> ess(LazyOpenInterpreter.java:110)
>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServ
>> er.getProgress(RemoteInterpreterServer.java:404)
>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>> ice$Processor$getProgress.getResult(RemoteInterpreterService.java:1509)
>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>> ice$Processor$getProgress.getResult(RemoteInterpreterService.java:1494)
>> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>> at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run
>> (TThreadPoolServer.java:285)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>> Executor.java:1142)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>> lExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>>
>>
>> *Zeppelin Log*
>> ERROR [2016-11-16 14:42:05,664] ({Thread-377}
>> JobProgressPoller.java[run]:54) - Can not get or update progress
>> org.apache.zeppelin.interpreter.InterpreterException:
>> org.apache.thrift.transport.TTransportException
>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.get
>> Progress(RemoteInterpreter.java:373)
>> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgr
>> ess(LazyOpenInterpreter.java:111)
>> at org.apache.zeppelin.notebook.Paragraph.progress(Paragraph.java:237)
>> at org.apache.zeppelin.scheduler.JobProgressPoller.run(JobProgr
>> essPoller.java:51)
>> Caused by: org.apache.thrift.transport.TTransportException
>> at org.apache.thrift.transport.TIOStreamTransport.read(TIOStrea
>> mTransport.java:132)
>> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
>> at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryPr
>> otocol.java:429)
>> at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryPr
>> otocol.java:318)
>> at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(
>> TBinaryProtocol.java:219)
>> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>> ice$Client.recv_getProgress(RemoteInterpreterService.java:296)
>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>> ice$Client.getProgress(RemoteInterpreterService.java:281)
>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.get
>> Progress(RemoteInterpreter.java:370)
>> ... 3 more
>>
>>
>>
>> When I tried run R commands using sparkR by launching
>> $SPARK_HOME/bin/sparkR, it works.
>>
>>
>> Would much appreciate any help.
>>
>> Thanks!
>> Abul
>>
>>
>> On Fri, Sep 16, 2016 at 5:36 PM, Hyung Sung Shim
>> wrote:
>>
>>> Hello.
>>> I think that `zeppelin.interpreter.host` and `zeppelin.interpreter.port`
>>> values on Interpreter setting UI should be removed because Interpreter
>>> will automatically be launched w
Re: FW: Issue with Zeppelin setup on Datastax-Spark
Ok I understand why you have issue.
You are using Spark 2.0.2 and latest Datastax 5.0.3 is still using Spark
version 1.6.X
On Wed, Nov 16, 2016 at 10:23 AM, Abul Basar wrote:
> I am facing a similar issue while using Spark R.
>
> My environment:
>
>- Spark 2.0.2
>- Zeppelin: 0.6.2
>- Java 1.8.0_111
>- R: 3.3.1
>
>
> SPARK_HOME is set. I am trying to run a simple paragraph.
>
> %r
> print("hello ...")
>
> I get the following exception
>
> *Interpreter Log*
> > # getZeppelinR
>
> > .zeppelinR = SparkR:::callJStatic("org.apache.zeppelin.spark.ZeppelinR",
> "getZeppelinR", hashCode)
>
> at org.apache.zeppelin.spark.ZeppelinR.waitForRScriptInitialize
> d(ZeppelinR.java:295)
> at org.apache.zeppelin.spark.ZeppelinR.request(ZeppelinR.java:235)
> at org.apache.zeppelin.spark.ZeppelinR.eval(ZeppelinR.java:183)
> at org.apache.zeppelin.spark.ZeppelinR.open(ZeppelinR.java:172)
> at org.apache.zeppelin.spark.SparkRInterpreter.open(SparkRInter
> preter.java:85)
> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(Laz
> yOpenInterpreter.java:69)
> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgr
> ess(LazyOpenInterpreter.java:110)
> at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServ
> er.getProgress(RemoteInterpreterServer.java:404)
> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
> ice$Processor$getProgress.getResult(RemoteInterpreterService.java:1509)
> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
> ice$Processor$getProgress.getResult(RemoteInterpreterService.java:1494)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.
> run(TThreadPoolServer.java:285)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> Executor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> lExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>
>
> *Zeppelin Log*
> ERROR [2016-11-16 14:42:05,664] ({Thread-377}
> JobProgressPoller.java[run]:54) - Can not get or update progress
> org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.thrift.transport.TTransportException
> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.
> getProgress(RemoteInterpreter.java:373)
> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgr
> ess(LazyOpenInterpreter.java:111)
> at org.apache.zeppelin.notebook.Paragraph.progress(Paragraph.java:237)
> at org.apache.zeppelin.scheduler.JobProgressPoller.run(JobProgr
> essPoller.java:51)
> Caused by: org.apache.thrift.transport.TTransportException
> at org.apache.thrift.transport.TIOStreamTransport.read(TIOStrea
> mTransport.java:132)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
> at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryPr
> otocol.java:429)
> at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryPr
> otocol.java:318)
> at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(
> TBinaryProtocol.java:219)
> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
> ice$Client.recv_getProgress(RemoteInterpreterService.java:296)
> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
> ice$Client.getProgress(RemoteInterpreterService.java:281)
> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.
> getProgress(RemoteInterpreter.java:370)
> ... 3 more
>
>
>
> When I tried run R commands using sparkR by launching
> $SPARK_HOME/bin/sparkR, it works.
>
>
> Would much appreciate any help.
>
> Thanks!
> Abul
>
>
> On Fri, Sep 16, 2016 at 5:36 PM, Hyung Sung Shim
> wrote:
>
>> Hello.
>> I think that `zeppelin.interpreter.host` and `zeppelin.interpreter.port`
>> values on Interpreter setting UI should be removed because Interpreter
>> will automatically be launched when you run a paragraph.
>>
>> 2016-09-16 3:56 GMT+09:00 Arpan Saha :
>>
>>> Yes. It is a zeppelin log. Not spark interpreter log. Can you please
>>> tell me where can I get spark interpreter log? I don't find anything under
>>> zeppelin/logs/ path.
>>>
>>> Thanks,
>>> Arpan
>>>
>>> From: DuyHai Doan [doanduy...@gmail.com]
>>> Sent: Thursday, September 15, 2016 11:20 AM
>>> To: users@zeppelin.apache.org
>>> Subject: Re: FW: Issue with Zeppelin setup on Datastax-Spark
>&
Re: FW: Issue with Zeppelin setup on Datastax-Spark
I am facing a similar issue while using Spark R.
My environment:
- Spark 2.0.2
- Zeppelin: 0.6.2
- Java 1.8.0_111
- R: 3.3.1
SPARK_HOME is set. I am trying to run a simple paragraph.
%r
print("hello ...")
I get the following exception
*Interpreter Log*
> # getZeppelinR
> .zeppelinR = SparkR:::callJStatic("org.apache.zeppelin.spark.ZeppelinR",
"getZeppelinR", hashCode)
at org.apache.zeppelin.spark.ZeppelinR.waitForRScriptInitialized(
ZeppelinR.java:295)
at org.apache.zeppelin.spark.ZeppelinR.request(ZeppelinR.java:235)
at org.apache.zeppelin.spark.ZeppelinR.eval(ZeppelinR.java:183)
at org.apache.zeppelin.spark.ZeppelinR.open(ZeppelinR.java:172)
at org.apache.zeppelin.spark.SparkRInterpreter.open(
SparkRInterpreter.java:85)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(
LazyOpenInterpreter.java:69)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgress(
LazyOpenInterpreter.java:110)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer.
getProgress(RemoteInterpreterServer.java:404)
at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$
Processor$getProgress.getResult(RemoteInterpreterService.java:1509)
at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$
Processor$getProgress.getResult(RemoteInterpreterService.java:1494)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(
TThreadPoolServer.java:285)
at java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
*Zeppelin Log*
ERROR [2016-11-16 14:42:05,664] ({Thread-377} JobProgressPoller.java[run]:54)
- Can not get or update progress
org.apache.zeppelin.interpreter.InterpreterException:
org.apache.thrift.transport.TTransportException
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getProgress(
RemoteInterpreter.java:373)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgress(
LazyOpenInterpreter.java:111)
at org.apache.zeppelin.notebook.Paragraph.progress(Paragraph.java:237)
at org.apache.zeppelin.scheduler.JobProgressPoller.run(
JobProgressPoller.java:51)
Caused by: org.apache.thrift.transport.TTransportException
at org.apache.thrift.transport.TIOStreamTransport.read(
TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
at org.apache.thrift.protocol.TBinaryProtocol.readAll(
TBinaryProtocol.java:429)
at org.apache.thrift.protocol.TBinaryProtocol.readI32(
TBinaryProtocol.java:318)
at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(
TBinaryProtocol.java:219)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$
Client.recv_getProgress(RemoteInterpreterService.java:296)
at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$
Client.getProgress(RemoteInterpreterService.java:281)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getProgress(
RemoteInterpreter.java:370)
... 3 more
When I tried run R commands using sparkR by launching
$SPARK_HOME/bin/sparkR, it works.
Would much appreciate any help.
Thanks!
Abul
On Fri, Sep 16, 2016 at 5:36 PM, Hyung Sung Shim wrote:
> Hello.
> I think that `zeppelin.interpreter.host` and `zeppelin.interpreter.port`
> values on Interpreter setting UI should be removed because Interpreter
> will automatically be launched when you run a paragraph.
>
> 2016-09-16 3:56 GMT+09:00 Arpan Saha :
>
>> Yes. It is a zeppelin log. Not spark interpreter log. Can you please tell
>> me where can I get spark interpreter log? I don't find anything under
>> zeppelin/logs/ path.
>>
>> Thanks,
>> Arpan
>>
>> From: DuyHai Doan [doanduy...@gmail.com]
>> Sent: Thursday, September 15, 2016 11:20 AM
>> To: users@zeppelin.apache.org
>> Subject: Re: FW: Issue with Zeppelin setup on Datastax-Spark
>>
>> Is it Zeppelin log or Spark interpreter log ? I guess we need also the
>> spark interpreter log to understand what is breaking Spark under the hood.
>> org.apache.thrift.transport.TTransportException is just technical
>> exception on inter-jvm communication
>>
>> On Thu, Sep 15, 2016 at 7:13 PM, Arpan Saha > <mailto:arpan.s...@mindtree.com>> wrote:
>>
>> Hi,
>>
>> Please find below full stack trace.
>>
>> ERROR [2016-09-15 16:49:15,117] ({pool-1-thread-2}
>> RemoteInterpreter.java[init]:182) - Failed to create interpreter:
>> org.apache.zeppelin.spark.SparkInterpreter
>> ERROR [2016-09-15 16:49:15,118] ({poo
Re: FW: Issue with Zeppelin setup on Datastax-Spark
Hello.
I think that `zeppelin.interpreter.host` and `zeppelin.interpreter.port`
values on Interpreter setting UI should be removed because Interpreter will
automatically be launched when you run a paragraph.
2016-09-16 3:56 GMT+09:00 Arpan Saha :
> Yes. It is a zeppelin log. Not spark interpreter log. Can you please tell
> me where can I get spark interpreter log? I don't find anything under
> zeppelin/logs/ path.
>
> Thanks,
> Arpan
>
> From: DuyHai Doan [doanduy...@gmail.com]
> Sent: Thursday, September 15, 2016 11:20 AM
> To: users@zeppelin.apache.org
> Subject: Re: FW: Issue with Zeppelin setup on Datastax-Spark
>
> Is it Zeppelin log or Spark interpreter log ? I guess we need also the
> spark interpreter log to understand what is breaking Spark under the hood.
> org.apache.thrift.transport.TTransportException is just technical
> exception on inter-jvm communication
>
> On Thu, Sep 15, 2016 at 7:13 PM, Arpan Saha mailto:arpan.s...@mindtree.com>> wrote:
>
> Hi,
>
> Please find below full stack trace.
>
> ERROR [2016-09-15 16:49:15,117] ({pool-1-thread-2}
> RemoteInterpreter.java[init]:182) - Failed to create interpreter:
> org.apache.zeppelin.spark.SparkInterpreter
> ERROR [2016-09-15 16:49:15,118] ({pool-1-thread-2} Job.java[run]:189) -
> Job failed
> org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.thrift.transport.TTransportException
> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(
> RemoteInterpreter.java:183)
> at org.apache.zeppelin.interpreter.remote.
> RemoteInterpreter.getFormType(RemoteInterpreter.java:328)
> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.
> getFormType(LazyOpenInterpreter.java:105)
> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.
> java:260)
> at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
> at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(
> RemoteScheduler.java:328)
> at java.util.concurrent.Executors$RunnableAdapter.
> call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ScheduledThreadPoolExecutor$
> ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at java.util.concurrent.ScheduledThreadPoolExecutor$
> ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.thrift.transport.TTransportException
> at org.apache.thrift.transport.TIOStreamTransport.read(
> TIOStreamTransport.java:132)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.
> java:86)
> at org.apache.thrift.protocol.TBinaryProtocol.readAll(
> TBinaryProtocol.java:429)
> at org.apache.thrift.protocol.TBinaryProtocol.readI32(
> TBinaryProtocol.java:318)
> at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(
> TBinaryProtocol.java:219)
> at org.apache.thrift.TServiceClient.receiveBase(
> TServiceClient.java:69)
> at org.apache.zeppelin.interpreter.thrift.
> RemoteInterpreterService$Client.recv_createInterpreter(
> RemoteInterpreterService.java:184)
> at org.apache.zeppelin.interpreter.thrift.
> RemoteInterpreterService$Client.createInterpreter(
> RemoteInterpreterService.java:168)
> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(
> RemoteInterpreter.java:172)
> ... 12 more
> ERROR [2016-09-15 16:49:15,118] ({Thread-18}
> RemoteInterpreterEventPoller.java[run]:88)
> - Can't get RemoteInterpreterEvent
>
> Thanks,
> Arpan.
>
>
> From: DuyHai Doan [doanduy...@gmail.com<mailto:doanduy...@gmail.com>]
> Sent: Thursday, September 15, 2016 4:29 AM
> To: users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>
> Subject: Re: Issue with Zeppelin setup on Datastax-Spark
>
> Can you provide the complete stack trace ?
>
> Also, worth reading my blog post about integrating Spark/Zeppelin/C* or
> DSE : http://www.doanduyhai.com/blog/?p=2325
>
> On Thu, Sep 15, 2016 at 5:17 AM, Arpan Saha mailto:arpan.s...@mindtree.com><mailto:arpan.s...@mindtree.com arpan.s...@mindtree.com>>> wrote:
> Hi Team,
>
> I am trying to integrate Zeppelin 0.6.0 with DataStax 4.8.8 (which has
> Spark 1.4.2). After I configured following properties in zeppelin-env.sh
> when I start zeppelin daemon it started
RE: FW: Issue with Zeppelin setup on Datastax-Spark
Yes. It is a zeppelin log. Not spark interpreter log. Can you please tell me
where can I get spark interpreter log? I don't find anything under
zeppelin/logs/ path.
Thanks,
Arpan
From: DuyHai Doan [doanduy...@gmail.com]
Sent: Thursday, September 15, 2016 11:20 AM
To: users@zeppelin.apache.org
Subject: Re: FW: Issue with Zeppelin setup on Datastax-Spark
Is it Zeppelin log or Spark interpreter log ? I guess we need also the spark
interpreter log to understand what is breaking Spark under the hood.
org.apache.thrift.transport.TTransportException is just technical exception on
inter-jvm communication
On Thu, Sep 15, 2016 at 7:13 PM, Arpan Saha
mailto:arpan.s...@mindtree.com>> wrote:
Hi,
Please find below full stack trace.
ERROR [2016-09-15 16:49:15,117] ({pool-1-thread-2}
RemoteInterpreter.java[init]:182) - Failed to create interpreter:
org.apache.zeppelin.spark.SparkInterpreter
ERROR [2016-09-15 16:49:15,118] ({pool-1-thread-2} Job.java[run]:189) - Job
failed
org.apache.zeppelin.interpreter.InterpreterException:
org.apache.thrift.transport.TTransportException
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:183)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:328)
at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:105)
at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:260)
at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
at
org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:328)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.thrift.transport.TTransportException
at
org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
at
org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
at
org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
at
org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
at
org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.recv_createInterpreter(RemoteInterpreterService.java:184)
at
org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.createInterpreter(RemoteInterpreterService.java:168)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:172)
... 12 more
ERROR [2016-09-15 16:49:15,118] ({Thread-18}
RemoteInterpreterEventPoller.java[run]:88) - Can't get RemoteInterpreterEvent
Thanks,
Arpan.
From: DuyHai Doan [doanduy...@gmail.com<mailto:doanduy...@gmail.com>]
Sent: Thursday, September 15, 2016 4:29 AM
To: users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>
Subject: Re: Issue with Zeppelin setup on Datastax-Spark
Can you provide the complete stack trace ?
Also, worth reading my blog post about integrating Spark/Zeppelin/C* or DSE :
http://www.doanduyhai.com/blog/?p=2325
On Thu, Sep 15, 2016 at 5:17 AM, Arpan Saha
mailto:arpan.s...@mindtree.com><mailto:arpan.s...@mindtree.com<mailto:arpan.s...@mindtree.com>>>
wrote:
Hi Team,
I am trying to integrate Zeppelin 0.6.0 with DataStax 4.8.8 (which has Spark
1.4.2). After I configured following properties in zeppelin-env.sh when I start
zeppelin daemon it started and in the browser I can see zeppelin is running but
when I am trying to execute spark query in the notebook it is throwing below
Error. Could you please help me to solve this issue.
export JAVA_HOME=
export SPARK_HOME=/etc/dse/spark
export HADOOP_CONF_DIR=/etc/dse/hadoop
export MASTER=spark://:7077
Also, added/updated below properties in the Spark Interpreter screen of
Zeppelin UI.
Master
spark.app.name<http://spark.app.name><http://spark.app.name>
spark.cassandra.auth.password
spark.cassandra.auth.username
spark.cassandra.connection.host
spark.cores.max
spark.executor.memory
zeppelin.interpreter.host
zeppelin.interpreter.port
Trying to execute below statement in the notebook:
%spark
sc
Re: FW: Issue with Zeppelin setup on Datastax-Spark
Is it Zeppelin log or Spark interpreter log ? I guess we need also the
spark interpreter log to understand what is breaking Spark under the hood.
org.apache.thrift.transport.TTransportException is just technical exception
on inter-jvm communication
On Thu, Sep 15, 2016 at 7:13 PM, Arpan Saha wrote:
>
> Hi,
>
> Please find below full stack trace.
>
> ERROR [2016-09-15 16:49:15,117] ({pool-1-thread-2}
> RemoteInterpreter.java[init]:182) - Failed to create interpreter:
> org.apache.zeppelin.spark.SparkInterpreter
> ERROR [2016-09-15 16:49:15,118] ({pool-1-thread-2} Job.java[run]:189) -
> Job failed
> org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.thrift.transport.TTransportException
> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(
> RemoteInterpreter.java:183)
> at org.apache.zeppelin.interpreter.remote.
> RemoteInterpreter.getFormType(RemoteInterpreter.java:328)
> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.
> getFormType(LazyOpenInterpreter.java:105)
> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.
> java:260)
> at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
> at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(
> RemoteScheduler.java:328)
> at java.util.concurrent.Executors$RunnableAdapter.
> call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ScheduledThreadPoolExecutor$
> ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at java.util.concurrent.ScheduledThreadPoolExecutor$
> ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.thrift.transport.TTransportException
> at org.apache.thrift.transport.TIOStreamTransport.read(
> TIOStreamTransport.java:132)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.
> java:86)
> at org.apache.thrift.protocol.TBinaryProtocol.readAll(
> TBinaryProtocol.java:429)
> at org.apache.thrift.protocol.TBinaryProtocol.readI32(
> TBinaryProtocol.java:318)
> at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(
> TBinaryProtocol.java:219)
> at org.apache.thrift.TServiceClient.receiveBase(
> TServiceClient.java:69)
> at org.apache.zeppelin.interpreter.thrift.
> RemoteInterpreterService$Client.recv_createInterpreter(
> RemoteInterpreterService.java:184)
> at org.apache.zeppelin.interpreter.thrift.
> RemoteInterpreterService$Client.createInterpreter(
> RemoteInterpreterService.java:168)
> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(
> RemoteInterpreter.java:172)
> ... 12 more
> ERROR [2016-09-15 16:49:15,118] ({Thread-18}
> RemoteInterpreterEventPoller.java[run]:88)
> - Can't get RemoteInterpreterEvent
>
> Thanks,
> Arpan.
>
>
> From: DuyHai Doan [doanduy...@gmail.com]
> Sent: Thursday, September 15, 2016 4:29 AM
> To: users@zeppelin.apache.org
> Subject: Re: Issue with Zeppelin setup on Datastax-Spark
>
> Can you provide the complete stack trace ?
>
> Also, worth reading my blog post about integrating Spark/Zeppelin/C* or
> DSE : http://www.doanduyhai.com/blog/?p=2325
>
> On Thu, Sep 15, 2016 at 5:17 AM, Arpan Saha mailto:arpan.s...@mindtree.com>> wrote:
> Hi Team,
>
> I am trying to integrate Zeppelin 0.6.0 with DataStax 4.8.8 (which has
> Spark 1.4.2). After I configured following properties in zeppelin-env.sh
> when I start zeppelin daemon it started and in the browser I can see
> zeppelin is running but when I am trying to execute spark query in the
> notebook it is throwing below Error. Could you please help me to solve this
> issue.
>
> export JAVA_HOME=
> export SPARK_HOME=/etc/dse/spark
> export HADOOP_CONF_DIR=/etc/dse/hadoop
> export MASTER=spark://:7077
>
> Also, added/updated below properties in the Spark Interpreter screen of
> Zeppelin UI.
>
> Master
> spark.app.name
> spark.cassandra.auth.password
> spark.cassandra.auth.username
> spark.cassandra.connection.host
> spark.cores.max
> spark.executor.memory
> zeppelin.interpreter.host
> zeppelin.interpreter.port
>
> Trying to execute below statement in the notebook:
> %spark
> sc.version
>
> Below Error is getting:
> java.net.SocketException: Connection reset at java.net.SocketInputStream.
> read(SocketInputStream.java:209) at java.net.SocketInputStream.
> read(SocketInputStream.java:141) at java.io.BufferedInputStream.
> fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.
> read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.
> read(BufferedInputStream.java:345) at org.apache.thrift.transport.
