Arjun J created ZEPPELIN-5741:
---------------------------------
Summary: unable to run spark job within Zeppelin 0.10.1
Key: ZEPPELIN-5741
URL: https://issues.apache.org/jira/browse/ZEPPELIN-5741
Project: Zeppelin
Issue Type: Bug
Components: spark, spark-submit, zeppelin-client, zeppelin-interpreter
Affects Versions: 0.10.1
Reporter: Arjun J
Fix For: 0.10.1
Attachments: Docker merged.txt, Zepp-Spark.jpg, spark master log.txt,
spark slave log.txt, spark.jpg, zepp log.txt
Hi,
I downloaded the Zeppelin docker file from the below site and then i took the
spark standalone docker file and merged them into one (attached). I was able to
create the image and run the cluster.
1) Zeppelin 0.10.1 : [https://hub.docker.com/r/apache/zeppelin/dockerfile]
2) Spark standalone :
[https://github.com/apache/zeppelin/blob/master/scripts/docker/spark-cluster-managers/spark_standalone/Dockerfile|https://github.com/apache/zeppelin/blob/master/scripts/docker/spark-cluster-managers/spark_standalone/Dockerfile.]
Command used to run the container :
docker run -it \
-p 8080:8080 \
-p 7077:7077 \
-p 8888:8888 \
-p 8081:8081 \
-h sparkmaster \
--name imagename\
imagename bash;
3) I am able to connect to Zeppelin Ui : http://hostname:8080/and run basic r/
python notebook but when I run a simple spark code it keeps running and does
not finish & its gets stuck. I have restart zep-daemeon everytime to test it
again. The application id doesn't even show in the spark master ui:
[http://hostname:8081/.|http://hostname:8081/]
4) I am able to connect to spark-shell, pyspark and do spark-submit within the
container and can see the application in the spark master url but none works
from Zeppelin. I have been struggling to make it run from past 1 week :(.
Please help.
Note : I had set the spark_home : /usr/local/spark/ and master :
spark://sparkmaster:7077 in the interpreter setting for spark but it didn't
help. There is also no interpreter log being generated within the zeppelin logs
folder.
I have attached the logs (spark master , slave and zepp logs) but no errors in
it.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
