Hi, I have a little problem in submitting our application to a mesos cluster.
Basically the mesos cluster is configured and I'm able to have spark-shell working correctly. Then I tried to launch our application jar (a uber, sbt assembly jar with all deps): bin/spark-submit --master mesos://10.192.222.232:5050 --jars $ADDITIONAL_JARS --class my.package.BenchmarkDriver file:///home/jclouds/spark-benchmark-assembly-0.1.0-SNAPSHOT.jar --application_configs So you can see I follow the documentation, provide the application jar in file://... format and I make sure that the application jar is available in this path in each worker of the cluster. Plus I provide the application jar with --jars. However there's always ClassNotFound exception in worker: java.lang.ClassNotFoundException: my.package.UrlLink or, if I tried custom kryo serializer: org.apache.spark.SparkException: Failed to invoke my.package.CustomKryoRegistrator I've tried using hdfs://... for the application jar, but it seems ignored completely by spark-submit. I'm using spark 1.1.1 on hadoop 2.4. Any suggestions? How should I submit the application jar? -- *JU Han* Data Engineer @ Botify.com +33 0619608888
