Can you try it with just: spark-submit --master spark://master:6066 --class SimpleApp target/simple-project-1.0.jar
And see if it works? Even better idea would be to spawn a spark-shell (*MASTER=spark://master:6066 bin/spark-shell*) and try out a simple *sc.parallelize(1 to 1000).collect* Thanks Best Regards On Wed, Oct 28, 2015 at 7:53 PM, Mark Bonnekessel <mar...@mailbox.org> wrote: > Hi, > > we are trying to setup apache spark on a raspberry pi cluster for > educational use. > Spark is installed in a docker container and all necessary ports are > exposed. > > After we start master and workers, all workers are listed as alive in the > master web ui (http://master:8080). > > I want to run the SimpleApp-Example from the spark-homepage ( > http://spark.apache.org/docs/latest/quick-start.html#self-contained-applications) > on > the cluster to verify that everything is working, but i cannot get it run. > > I built an jar-file and submitted the application with spark-submit and > get the following output: > spark-submit --master spark://master:6066 --deploy-mode cluster --class > SimpleApp target/simple-project-1.0.jar > Running Spark using the REST application submission protocol. > Using Spark's default log4j profile: > org/apache/spark/log4j-defaults.properties > 15/10/28 12:54:43 INFO RestSubmissionClient: Submitting a request to > launch an application in spark://localhost:6066. > 15/10/28 12:54:43 INFO RestSubmissionClient: Submission successfully > created as driver-20151028115443-0002. Polling submission state... > 15/10/28 12:54:43 INFO RestSubmissionClient: Submitting a request for the > status of submission driver-20151028115443-0002 in spark://localhost:6066. > 15/10/28 12:54:43 INFO RestSubmissionClient: State of driver > driver-20151028115443-0002 is now SUBMITTED. > 15/10/28 12:54:43 INFO RestSubmissionClient: Server responded with > CreateSubmissionResponse: > { > "action" : "CreateSubmissionResponse", > "message" : "Driver successfully submitted as > driver-20151028115443-0002", > "serverSparkVersion" : "1.5.1", > "submissionId" : "driver-20151028115443-0002", > "success" : true > } > > The driver is created correctly, but it never starts the application. > What am i missing? > > Regards, > Mark >
