Hi Jeff,
PFB the interpreter log.
INFO [2018-01-03 12:10:05,960] ({pool-2-thread-9}
Logging.scala[logInfo]:58) - Starting HTTP Server
INFO [2018-01-03 12:10:05,961] ({pool-2-thread-9}
Server.java[doStart]:272) - jetty-8.y.z-SNAPSHOT
INFO [2018-01-03 12:10:05,963] ({pool-2-thread-9}
AbstractConnector.java[doStart]:338) - Started SocketConnector@0.0.0.0:58989
INFO [2018-01-03 12:10:05,963] ({pool-2-thread-9}
Logging.scala[logInfo]:58) - Successfully started service 'HTTP class
server' on port 58989.
INFO [2018-01-03 12:10:06,094] ({dispatcher-event-loop-1}
Logging.scala[logInfo]:58) - Removed broadcast_1_piece0 on localhost:42453
in memory (size: 854.0 B, free: 511.1 MB)
INFO [2018-01-03 12:10:07,049] ({pool-2-thread-9}
ZeppelinR.java[createRScript]:353) - File
/tmp/zeppelin_sparkr-5046601627391341672.R created
ERROR [2018-01-03 12:10:17,051] ({pool-2-thread-9} Job.java[run]:188) - Job
failed
*org.apache.zeppelin.interpreter.InterpreterException: sparkr is not
responding *
R version 3.4.1 (2017-06-30) -- "Single Candle"
Copyright (C) 2017 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
....
....
> args <- commandArgs(trailingOnly = TRUE)
> hashCode <- as.integer(args[1])
> port <- as.integer(args[2])
> libPath <- args[3]
> version <- as.integer(args[4])
> rm(args)
>
> print(paste("Port ", toString(port)))
[1]
"Port 58063"
> print(paste("LibPath ", libPath))
[1]
"LibPath /home/meethu/spark-1.6.1-bin-hadoop2.6/R/lib"
>
> .libPaths(c(file.path(libPath), .libPaths()))
> library(SparkR)
Attaching package: ‘SparkR’
The following objects are masked from ‘package:stats’:
cov, filter, lag, na.omit, predict, sd, var
The following objects are masked from ‘package:base’:
colnames, colnames<-, endsWith, intersect, rank, rbind, sample,
startsWith, subset, summary, table, transform
> SparkR:::connectBackend("localhost", port, 6000)
A connection with
description "->localhost:58063"
class
"sockconn"
mode "wb"
text "binary"
opened "opened"
can read "yes"
can write "yes"
>
> # scStartTime is needed by R/pkg/R/sparkR.R
> assign(".scStartTime", as.integer(Sys.time()), envir =
SparkR:::.sparkREnv)
> # getZeppelinR
> *.zeppelinR = SparkR:::callJStatic("org.apache.zeppelin.spark.ZeppelinR",
"getZeppelinR", hashCode)*
at
org.apache.zeppelin.spark.ZeppelinR.waitForRScriptInitialized(ZeppelinR.java:285)
at org.apache.zeppelin.spark.ZeppelinR.request(ZeppelinR.java:227)
at org.apache.zeppelin.spark.ZeppelinR.eval(ZeppelinR.java:176)
at org.apache.zeppelin.spark.ZeppelinR.open(ZeppelinR.java:165)
at
org.apache.zeppelin.spark.SparkRInterpreter.open(SparkRInterpreter.java:90)
at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:70)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:491)
at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
INFO [2018-01-03 12:10:17,070] ({pool-2-thread-9}
SchedulerFactory.java[jobFinished]:137) - Job
remoteInterpretJob_1514961605951 finished by scheduler
org.apache.zeppelin.spark.SparkRInterpreter392022746
INFO [2018-01-03 12:39:22,664] ({Spark Context Cleaner}
Logging.scala[logInfo]:58) - Cleaned accumulator 2
PFB the output of the command *ps -ef | grep /usr/lib/R/bin/exec/R*
meethu 6647 6470 0 12:09 pts/1 00:00:00 /usr/lib/R/bin/exec/R
--no-save --no-restore -f /tmp/zeppelin_sparkr-1100854828050763213.R --args
214655664 58063 /home/meethu/spark-1.6.1-bin-hadoop2.6/R/lib 10601
meethu 6701 6470 0 12:09 pts/1 00:00:00 /usr/lib/R/bin/exec/R
--no-save --no-restore -f /tmp/zeppelin_sparkr-4152305170353311178.R --args
1642312173 58063 /home/meethu/spark-1.6.1-bin-hadoop2.6/R/lib 10601
meethu 6745 6470 0 12:10 pts/1 00:00:00 /usr/lib/R/bin/exec/R
--no-save --no-restore -f /tmp/zeppelin_sparkr-5046601627391341672.R --args
1158632477 58063 /home/meethu/spark-1.6.1-bin-hadoop2.6/R/lib 10601
Regards,
Meethu Mathew
On Wed, Jan 3, 2018 at 12:56 PM, Jeff Zhang <zjf...@gmail.com> wrote:
>
> Could you check the interpreter log ?
>
> Meethu Mathew <meethu.mat...@flytxt.com>于2018年1月3日周三 下午3:05写道：
>
>> Hi,
>>
>> I have met with a strange issue in running R notebooks in zeppelin(0.7.2).
>> Spark intrepreter is in per note Scoped mode and spark version is 1.6.2
>>
>> Please find the steps below to reproduce the issue:
>> 1. Create a notebook (Note1) and run any r code in a paragraph. I ran the
>> following code.
>>
>>> %r
>>>
>>> rdf <- data.frame(c(1,2,3,4))
>>>
>>> colnames(rdf) <- c("myCol")
>>>
>>> sdf <- createDataFrame(sqlContext, rdf)
>>>
>>> withColumn(sdf, "newCol", sdf$myCol * 2.0)
>>>
>>>
>> 2. Create another notebook (Note2) and run any r code in a paragraph. I
>> ran the same code as above.
>>
>> Till now everything works fine.
>>
>> 3. Create third notebook (Note3) and run any r code in a paragraph. I
>> ran the same code. This notebook fails with the error
>>
>>> org.apache.zeppelin.interpreter.InterpreterException: sparkr is not
>>> responding
>>
>>
>> What I understood from the analysis is that the process created for
>> sparkr interpreter is not getting killed properly and this makes every
>> third model to throw an error while executing. The process will be killed
>> on restarting the sparkr interpreter and another 2 models could be executed
>> successfully. ie, For every third model run using the sparkr interpreter,
>> the error is thrown. We suspect this as a limitation with zeppelin.
>>
>> Please help to solve this issue
>>
>> Regards,
>>
>>
>> Meethu Mathew
>>
>>
