ps aux | grep RemoteInterpreterServer root 71 0.1 3.3 3528164 133668 ? Sl 06:21 0:04 /usr/lib/jvm/java-8-openjdk-amd64//bin/java -Dfile.encoding=UTF-8 -Dlog4j.configuration=file:///opt/zeppelin-0.6.1-bin-all/conf/log4j.properties -Dzeppelin.log.file=/opt/zeppelin-0.6.1-bin-all/logs/zeppelin-interpreter-sh--cad975db8a21.log -Xms1024m -Xmx1024m -XX:MaxPermSize=512m -cp ::/opt/zeppelin-0.6.1-bin-all/interpreter/sh/*::/opt/zeppelin-0.6.1-bin-all/lib/zeppelin-interpreter-0.6.1.jar org.apache.zeppelin.interpreter.remote.*RemoteInterpreterServer *45877 root 346 0.7 13.7 3612088 555092 ? Sl 06:28 0:18 /usr/lib/jvm/java-8-openjdk-amd64//bin/java -cp /opt/zeppelin-0.6.1-bin-all/interpreter/spark/*:/opt/zeppelin-0.6.1-bin-all/lib/zeppelin-interpreter-0.6.1.jar:/opt/zeppelin-0.6.1-bin-all/interpreter/spark/zeppelin-spark_2.11-0.6.1.jar:/opt/spark/conf/:/opt/spark/jars/*:/opt/hadoop/conf/ -Xmx1g -Dfile.encoding=UTF-8 -Dlog4j.configuration=file:///opt/zeppelin-0.6.1-bin-all/conf/log4j.properties -Dzeppelin.log.file=/opt/zeppelin-0.6.1-bin-all/logs/zeppelin-interpreter-spark--cad975db8a21.log org.apache.spark.deploy.SparkSubmit --conf spark.driver.extraClassPath=::/opt/zeppelin-0.6.1-bin-all/interpreter/spark/*::/opt/zeppelin-0.6.1-bin-all/lib/zeppelin-interpreter-0.6.1.jar:/opt/zeppelin-0.6.1-bin-all/interpreter/spark/zeppelin-spark_2.11-0.6.1.jar --conf spark.driver.extraJavaOptions= -Dfile.encoding=UTF-8 -Dlog4j.configuration=file:///opt/zeppelin-0.6.1-bin-all/conf/log4j.properties -Dzeppelin.log.file=/opt/zeppelin-0.6.1-bin-all/logs/zeppelin-interpreter-spark--cad975db8a21.log --class org.apache.zeppelin.interpreter.remote.*RemoteInterpreterServer */opt/zeppelin-0.6.1-bin-all/interpreter/spark/zeppelin-spark_2.11-0.6.1.jar 44320
I started my Zeppelin instance with /opt/zeppelin-0.6.1-bin-all/bin/zeppelin.sh --conf /opt/zeppelin-0.6.1-bin-all/conf On Sun, Oct 9, 2016 at 2:54 PM Jeff Zhang <zjf...@gmail.com> wrote: > Could you check the classpath of the interpreter process ? I suspect you > didn't export HADOOP_CONF_DIR correctly. > > Run the following command: > > ps aux | grep RemoteInterpreterServer > > On Sun, Oct 9, 2016 at 2:36 PM, Xi Shen <davidshe...@gmail.com> wrote: > > Hi, > > I followed http://zeppelin.apache.org/docs/latest/interpreter/spark.html, > and set up SPARK_HOME, HADOOP_CONF_DIR. > > My SPARK build is 2.0. My Zeppelin build is the 0.6.1 binary from the web. > > After I start Zeppelin, I went to the interpreter setting page and changed > Spark interpreter settings as follow: > > master: yarn > deploy-mode: client > > Then, I created a new notebook and executed: > > %spark > > spark.version > > The block never finishes...no error either. > > In the ./logs/zeppelin-interpreter-spark-*.log, I found the following *which > I think is the cause of my problem* > > INFO [2016-10-09 06:28:37,074] ({pool-2-thread-2} > Logging.scala[logInfo]:54) - Added JAR > *file:/opt/zeppelin-0.6.1-bin-all/interpreter/spark/zeppelin-spark_2.11-0.6.1.jar* > at spark://172.17.0.3:38775/jars/zeppelin-spark_2.11-0.6.1.jar with > timestamp 1475994517073 > INFO [2016-10-09 06:28:37,150] ({pool-2-thread-2} > Logging.scala[logInfo]:54) - Created default pool default, schedulingMode: > FIFO, minShare: 0, weight: 1 > INFO [2016-10-09 06:28:38,205] ({pool-2-thread-2} > RMProxy.java[createRMProxy]:98) - *Connecting to ResourceManager at > /0.0.0.0:8032 <http://0.0.0.0:8032>* > > Looks like Zeppelin is not using my Spark binary, and is not using my > hadoop configuration. > > What did I miss? > > -- > > > Thanks, > David S. > > > > > -- > Best Regards > > Jeff Zhang > -- Thanks, David S.
