Yang Lei created ZEPPELIN-120:
---------------------------------
Summary: Tutorial fails on Mesos master
Key: ZEPPELIN-120
URL: https://issues.apache.org/jira/browse/ZEPPELIN-120
Project: Zeppelin
Issue Type: Bug
Components: Interpreters
Reporter: Yang Lei
I was able to run tutorial on spark local mode (w/o MASTER environment
variable).
However when running on Mesos , the %sql script failed with
"java.lang.reflect.InvocationTargetException", detailed exception as follows:
Exception in thread "Thread-83" org.apache.spark.SparkException: Executor Spark
home `spark.mesos.executor.home` is not set!
at
org.apache.spark.scheduler.cluster.mesos.MesosSchedulerBackend$$anonfun$2.apply(MesosSchedulerBackend.scala:74)
at
org.apache.spark.scheduler.cluster.mesos.MesosSchedulerBackend$$anonfun$2.apply(MesosSchedulerBackend.scala:74)
at scala.Option.getOrElse(Option.scala:120)
at
org.apache.spark.scheduler.cluster.mesos.MesosSchedulerBackend.createExecutorInfo(MesosSchedulerBackend.scala:73)
at
org.apache.spark.scheduler.cluster.mesos.MesosSchedulerBackend.createMesosTask(MesosSchedulerBackend.scala:270)
at
org.apache.spark.scheduler.cluster.mesos.MesosSchedulerBackend$$anonfun$resourceOffers$1$$anonfun$apply$mcV$sp$2$$anonfun$apply$1.apply(MesosSchedulerBackend.scala:232)
at
org.apache.spark.scheduler.cluster.mesos.MesosSchedulerBackend$$anonfun$resourceOffers$1$$anonfun$apply$mcV$sp$2$$anonfun$apply$1.apply(MesosSchedulerBackend.scala:226)
at
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
at
org.apache.spark.scheduler.cluster.mesos.MesosSchedulerBackend$$anonfun$resourceOffers$1$$anonfun$apply$mcV$sp$2.apply(MesosSchedulerBackend.scala:226)
at
org.apache.spark.scheduler.cluster.mesos.MesosSchedulerBackend$$anonfun$resourceOffers$1$$anonfun$apply$mcV$sp$2.apply(MesosSchedulerBackend.scala:225)
at
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
at
org.apache.spark.scheduler.cluster.mesos.MesosSchedulerBackend$$anonfun$resourceOffers$1.apply$mcV$sp(MesosSchedulerBackend.scala:225)
at
org.apache.spark.scheduler.cluster.mesos.MesosSchedulerBackend.inClassLoader(MesosSchedulerBackend.scala:172)
at
org.apache.spark.scheduler.cluster.mesos.MesosSchedulerBackend.resourceOffers(MesosSchedulerBackend.scala:188)
I defined the following in conf/zeppelin-env.sh
export MESOS_NATIVE_JAVA_LIBRARY=/usr/local/lib/libmesos.so
export
ZEPPELIN_JAVA_OPTS="-Dspark.executor.uri=http://www.apache.org/dyn/closer.cgi/spark/spark-1.4.0/spark-1.4.0-bin-hadoop2.4.tgz";
export
SPARK_EXECUTOR_URI=http://www.apache.org/dyn/closer.cgi/spark/spark-1.4.0/spark-1.4.0-bin-hadoop2.4.tgz
I built the zeppelin with :
mvn clean package -Pspark-1.4 -Dhadoop.version=2.2.0 -Phadoop-2.2 -DskipTests
Before the failure,I can see zeppelin framework created on Mesos.I can also see
from the zeppelin application UI (:4040) that the configurations of
spark.executor.ui and spark.master are correct.
After the failure, the framework is terminated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
