Try passing jar using --jars option On Wed, Mar 2, 2016 at 10:17 AM Ashish Soni <asoni.le...@gmail.com> wrote:
> I made some progress but now i am stuck at this point , Please help as > looks like i am close to get it working > > I have everything running in docker container including mesos slave and > master > > When i try to submit the pi example i get below error > *Error: Cannot load main class from JAR file:/opt/spark/Example* > > Below is the command i use to submit as a docker container > > docker run -it --rm -e SPARK_MASTER="mesos://10.0.2.15:7077" -e > SPARK_IMAGE="spark_driver:latest" spark_driver:latest ./bin/spark-submit > --deploy-mode cluster --name "PI Example" --class > org.apache.spark.examples.SparkPi --driver-memory 512m --executor-memory > 512m --executor-cores 1 > http://10.0.2.15/spark-examples-1.6.0-hadoop2.6.0.jar > > > On Tue, Mar 1, 2016 at 2:59 PM, Timothy Chen <t...@mesosphere.io> wrote: > >> Can you go through the Mesos UI and look at the driver/executor log from >> steer file and see what the problem is? >> >> Tim >> >> On Mar 1, 2016, at 8:05 AM, Ashish Soni <asoni.le...@gmail.com> wrote: >> >> Not sure what is the issue but i am getting below error when i try to >> run spark PI example >> >> Blacklisting Mesos slave value: "5345asdasdasdkas234234asdasdasdasd" >> due to too many failures; is Spark installed on it? >> WARN TaskSchedulerImpl: Initial job has not accepted any resources; >> check your cluster UI to ensure that workers are registered and have >> sufficient resources >> >> >> On Mon, Feb 29, 2016 at 1:39 PM, Sathish Kumaran Vairavelu < >> vsathishkuma...@gmail.com> wrote: >> >>> May be the Mesos executor couldn't find spark image or the constraints >>> are not satisfied. Check your Mesos UI if you see Spark application in the >>> Frameworks tab >>> >>> On Mon, Feb 29, 2016 at 12:23 PM Ashish Soni <asoni.le...@gmail.com> >>> wrote: >>> >>>> What is the Best practice , I have everything running as docker >>>> container in single host ( mesos and marathon also as docker container ) >>>> and everything comes up fine but when i try to launch the spark shell i >>>> get below error >>>> >>>> >>>> SQL context available as sqlContext. >>>> >>>> scala> val data = sc.parallelize(1 to 100) >>>> data: org.apache.spark.rdd.RDD[Int] = ParallelCollectionRDD[0] at >>>> parallelize at <console>:27 >>>> >>>> scala> data.count >>>> [Stage 0:> (0 >>>> + 0) / 2]16/02/29 18:21:12 WARN TaskSchedulerImpl: Initial job has not >>>> accepted any resources; check your cluster UI to ensure that workers are >>>> registered and have sufficient resources >>>> 16/02/29 18:21:27 WARN TaskSchedulerImpl: Initial job has not accepted >>>> any resources; check your cluster UI to ensure that workers are registered >>>> and have sufficient resources >>>> >>>> >>>> >>>> On Mon, Feb 29, 2016 at 12:04 PM, Tim Chen <t...@mesosphere.io> wrote: >>>> >>>>> No you don't have to run Mesos in docker containers to run Spark in >>>>> docker containers. >>>>> >>>>> Once you have Mesos cluster running you can then specfiy the Spark >>>>> configurations in your Spark job (i.e: >>>>> spark.mesos.executor.docker.image=mesosphere/spark:1.6) >>>>> and Mesos will automatically launch docker containers for you. >>>>> >>>>> Tim >>>>> >>>>> On Mon, Feb 29, 2016 at 7:36 AM, Ashish Soni <asoni.le...@gmail.com> >>>>> wrote: >>>>> >>>>>> Yes i read that and not much details here. >>>>>> >>>>>> Is it true that we need to have spark installed on each mesos docker >>>>>> container ( master and slave ) ... >>>>>> >>>>>> Ashish >>>>>> >>>>>> On Fri, Feb 26, 2016 at 2:14 PM, Tim Chen <t...@mesosphere.io> wrote: >>>>>> >>>>>>> https://spark.apache.org/docs/latest/running-on-mesos.html should >>>>>>> be the best source, what problems were you running into? >>>>>>> >>>>>>> Tim >>>>>>> >>>>>>> On Fri, Feb 26, 2016 at 11:06 AM, Yin Yang <yy201...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Have you read this ? >>>>>>>> https://spark.apache.org/docs/latest/running-on-mesos.html >>>>>>>> >>>>>>>> On Fri, Feb 26, 2016 at 11:03 AM, Ashish Soni < >>>>>>>> asoni.le...@gmail.com> wrote: >>>>>>>> >>>>>>>>> Hi All , >>>>>>>>> >>>>>>>>> Is there any proper documentation as how to run spark on mesos , I >>>>>>>>> am trying from the last few days and not able to make it work. >>>>>>>>> >>>>>>>>> Please help >>>>>>>>> >>>>>>>>> Ashish >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >> >
