You shouldn't need to specify --jars at all since you only have one jar. The error is pretty odd as it suggests it's trying to load /opt/spark/Example but that doesn't really seem to be anywhere in your image or command.
Can you paste your stdout from the driver task launched by the cluster dispatcher, that shows you the spark-submit command it eventually ran? Tim On Wed, Mar 2, 2016 at 5:42 PM, Ashish Soni <asoni.le...@gmail.com> wrote: > See below and Attached the Dockerfile to build the spark image ( between > i just upgraded to 1.6 ) > > I am running below setup - > > Mesos Master - Docker Container > Mesos Slave 1 - Docker Container > Mesos Slave 2 - Docker Container > Marathon - Docker Container > Spark MESOS Dispatcher - Docker Container > > when i submit the Spark PI Example Job using below command > > *docker run -it --rm -m 2g -e SPARK_MASTER="mesos://10.0.2.15:7077 > <http://10.0.2.15:7077>" -e SPARK_IMAGE="spark_driver:**latest" > spark_driver:latest ./bin/spark-submit --deploy-mode cluster --name "PI > Example" --class org.apache.spark.examples.**SparkPi > http://10.0.2.15/spark-examples-1.6.0-hadoop2.6.0.jar > <http://10.0.2.15/spark-examples-1.6.0-hadoop2.6.0.jar> --jars > /opt/spark/lib/spark-examples-**1.6.0-hadoop2.6.0.jar --verbose* > > Below is the ERROR > Error: Cannot load main class from JAR file:/opt/spark/Example > Run with --help for usage help or --verbose for debug output > > > When i docker Inspect for the stopped / dead container i see below output > what is interesting to see is some one or executor replaced by original > command with below in highlighted and i do not see Executor is downloading > the JAR -- IS this a BUG i am hitting or not sure if that is supposed to > work this way and i am missing some configuration > > "Env": [ > "SPARK_IMAGE=spark_driver:latest", > "SPARK_SCALA_VERSION=2.10", > "SPARK_VERSION=1.6.0", > "SPARK_EXECUTOR_URI= > http://d3kbcqa49mib13.cloudfront.net/spark-1.6.0-bin-hadoop2.6.tgz";, > "MESOS_NATIVE_JAVA_LIBRARY=/usr/lib/libmesos-0.25.0.so", > "SPARK_MASTER=mesos://10.0.2.15:7077", > > "SPARK_EXECUTOR_OPTS=-Dspark.executorEnv.MESOS_NATIVE_JAVA_LIBRARY=/usr/lib/ > libmesos-0.25.0.so -Dspark.jars= > http://10.0.2.15/spark-examples-1.6.0-hadoop2.6.0.jar > -Dspark.mesos.mesosExecutor.cores=0.1 -Dspark.driver.supervise=false - > Dspark.app.name=PI Example -Dspark.mesos.uris= > http://10.0.2.15/spark-examples-1.6.0-hadoop2.6.0.jar > -Dspark.mesos.executor.docker.image=spark_driver:latest > -Dspark.submit.deployMode=cluster -Dspark.master=mesos://10.0.2.15:7077 > -Dspark.driver.extraClassPath=/opt/spark/custom/lib/* > -Dspark.executor.extraClassPath=/opt/spark/custom/lib/* > -Dspark.executor.uri= > http://d3kbcqa49mib13.cloudfront.net/spark-1.6.0-bin-hadoop2.6.tgz > -Dspark.mesos.executor.home=/opt/spark", > "MESOS_SANDBOX=/mnt/mesos/sandbox", > > "MESOS_CONTAINER_NAME=mesos-e47f8d4c-5ee1-4d01-ad07-0d9a03ced62d-S1.43c08f82-e508-4d57-8c0b-fa05bee77fd6", > > "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", > "HADOOP_VERSION=2.6", > "SPARK_HOME=/opt/spark" > ], > "Cmd": [ > "-c", > * "./bin/spark-submit --name PI Example --master > mesos://10.0.2.15:5050 <http://10.0.2.15:5050> --driver-cores 1.0 > --driver-memory 1024M --class org.apache.spark.examples.SparkPi > $MESOS_SANDBOX/spark-examples-1.6.0-hadoop2.6.0.jar --jars > /opt/spark/lib/spark-examples-1.6.0-hadoop2.6.0.jar --verbose"* > ], > "Image": "spark_driver:latest", > > > > > > > > > > > > > On Wed, Mar 2, 2016 at 5:49 PM, Charles Allen < > charles.al...@metamarkets.com> wrote: > >> @Tim yes, this is asking about 1.5 though >> >> On Wed, Mar 2, 2016 at 2:35 PM Tim Chen <t...@mesosphere.io> wrote: >> >>> Hi Charles, >>> >>> I thought that's fixed with your patch in latest master now right? >>> >>> Ashish, yes please give me your docker image name (if it's in the public >>> registry) and what you've tried and I can see what's wrong. I think it's >>> most likely just the configuration of where the Spark home folder is in the >>> image. >>> >>> Tim >>> >>> On Wed, Mar 2, 2016 at 2:28 PM, Charles Allen < >>> charles.al...@metamarkets.com> wrote: >>> >>>> Re: Spark on Mesos.... Warning regarding disk space: >>>> https://issues.apache.org/jira/browse/SPARK-12330 >>>> >>>> That's a spark flaw I encountered on a very regular basis on mesos. >>>> That and a few other annoyances are fixed in >>>> https://github.com/metamx/spark/tree/v1.5.2-mmx >>>> >>>> Here's another mild annoyance I've encountered: >>>> https://issues.apache.org/jira/browse/SPARK-11714 >>>> >>>> On Wed, Mar 2, 2016 at 1:31 PM Ashish Soni <asoni.le...@gmail.com> >>>> wrote: >>>> >>>>> I have no luck and i would to ask the question to spark committers >>>>> will this be ever designed to run on mesos ? >>>>> >>>>> spark app as a docker container not working at all on mesos ,if any >>>>> one would like the code i can send it over to have a look. >>>>> >>>>> Ashish >>>>> >>>>> On Wed, Mar 2, 2016 at 12:23 PM, Sathish Kumaran Vairavelu < >>>>> vsathishkuma...@gmail.com> wrote: >>>>> >>>>>> Try passing jar using --jars option >>>>>> >>>>>> On Wed, Mar 2, 2016 at 10:17 AM Ashish Soni <asoni.le...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> I made some progress but now i am stuck at this point , Please help >>>>>>> as looks like i am close to get it working >>>>>>> >>>>>>> I have everything running in docker container including mesos slave >>>>>>> and master >>>>>>> >>>>>>> When i try to submit the pi example i get below error >>>>>>> *Error: Cannot load main class from JAR file:/opt/spark/Example* >>>>>>> >>>>>>> Below is the command i use to submit as a docker container >>>>>>> >>>>>>> docker run -it --rm -e SPARK_MASTER="mesos://10.0.2.15:7077" -e >>>>>>> SPARK_IMAGE="spark_driver:latest" spark_driver:latest ./bin/spark-submit >>>>>>> --deploy-mode cluster --name "PI Example" --class >>>>>>> org.apache.spark.examples.SparkPi --driver-memory 512m --executor-memory >>>>>>> 512m --executor-cores 1 >>>>>>> http://10.0.2.15/spark-examples-1.6.0-hadoop2.6.0.jar >>>>>>> >>>>>>> >>>>>>> On Tue, Mar 1, 2016 at 2:59 PM, Timothy Chen <t...@mesosphere.io> >>>>>>> wrote: >>>>>>> >>>>>>>> Can you go through the Mesos UI and look at the driver/executor log >>>>>>>> from steer file and see what the problem is? >>>>>>>> >>>>>>>> Tim >>>>>>>> >>>>>>>> On Mar 1, 2016, at 8:05 AM, Ashish Soni <asoni.le...@gmail.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>> Not sure what is the issue but i am getting below error when i try >>>>>>>> to run spark PI example >>>>>>>> >>>>>>>> Blacklisting Mesos slave value: "5345asdasdasdkas234234asdasdasdasd" >>>>>>>> due to too many failures; is Spark installed on it? >>>>>>>> WARN TaskSchedulerImpl: Initial job has not accepted any >>>>>>>> resources; check your cluster UI to ensure that workers are registered >>>>>>>> and have sufficient resources >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Feb 29, 2016 at 1:39 PM, Sathish Kumaran Vairavelu < >>>>>>>> vsathishkuma...@gmail.com> wrote: >>>>>>>> >>>>>>>>> May be the Mesos executor couldn't find spark image or the >>>>>>>>> constraints are not satisfied. Check your Mesos UI if you see Spark >>>>>>>>> application in the Frameworks tab >>>>>>>>> >>>>>>>>> On Mon, Feb 29, 2016 at 12:23 PM Ashish Soni < >>>>>>>>> asoni.le...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> What is the Best practice , I have everything running as docker >>>>>>>>>> container in single host ( mesos and marathon also as docker >>>>>>>>>> container ) >>>>>>>>>> and everything comes up fine but when i try to launch the spark >>>>>>>>>> shell i >>>>>>>>>> get below error >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> SQL context available as sqlContext. >>>>>>>>>> >>>>>>>>>> scala> val data = sc.parallelize(1 to 100) >>>>>>>>>> data: org.apache.spark.rdd.RDD[Int] = ParallelCollectionRDD[0] at >>>>>>>>>> parallelize at <console>:27 >>>>>>>>>> >>>>>>>>>> scala> data.count >>>>>>>>>> [Stage 0:> >>>>>>>>>> (0 + 0) / 2]16/02/29 18:21:12 WARN TaskSchedulerImpl: Initial job >>>>>>>>>> has >>>>>>>>>> not accepted any resources; check your cluster UI to ensure that >>>>>>>>>> workers >>>>>>>>>> are registered and have sufficient resources >>>>>>>>>> 16/02/29 18:21:27 WARN TaskSchedulerImpl: Initial job has not >>>>>>>>>> accepted any resources; check your cluster UI to ensure that workers >>>>>>>>>> are >>>>>>>>>> registered and have sufficient resources >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Mon, Feb 29, 2016 at 12:04 PM, Tim Chen <t...@mesosphere.io> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> No you don't have to run Mesos in docker containers to run Spark >>>>>>>>>>> in docker containers. >>>>>>>>>>> >>>>>>>>>>> Once you have Mesos cluster running you can then specfiy the >>>>>>>>>>> Spark configurations in your Spark job (i.e: >>>>>>>>>>> spark.mesos.executor.docker.image=mesosphere/spark:1.6) >>>>>>>>>>> and Mesos will automatically launch docker containers for you. >>>>>>>>>>> >>>>>>>>>>> Tim >>>>>>>>>>> >>>>>>>>>>> On Mon, Feb 29, 2016 at 7:36 AM, Ashish Soni < >>>>>>>>>>> asoni.le...@gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Yes i read that and not much details here. >>>>>>>>>>>> >>>>>>>>>>>> Is it true that we need to have spark installed on each mesos >>>>>>>>>>>> docker container ( master and slave ) ... >>>>>>>>>>>> >>>>>>>>>>>> Ashish >>>>>>>>>>>> >>>>>>>>>>>> On Fri, Feb 26, 2016 at 2:14 PM, Tim Chen <t...@mesosphere.io> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> https://spark.apache.org/docs/latest/running-on-mesos.html should >>>>>>>>>>>>> be the best source, what problems were you running into? >>>>>>>>>>>>> >>>>>>>>>>>>> Tim >>>>>>>>>>>>> >>>>>>>>>>>>> On Fri, Feb 26, 2016 at 11:06 AM, Yin Yang <yy201...@gmail.com >>>>>>>>>>>>> > wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Have you read this ? >>>>>>>>>>>>>> https://spark.apache.org/docs/latest/running-on-mesos.html >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Fri, Feb 26, 2016 at 11:03 AM, Ashish Soni < >>>>>>>>>>>>>> asoni.le...@gmail.com> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi All , >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Is there any proper documentation as how to run spark on >>>>>>>>>>>>>>> mesos , I am trying from the last few days and not able to make >>>>>>>>>>>>>>> it work. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Please help >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Ashish >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>> >
