Hi, I am trying to submit a Spark job on Mesos using spark-submit from my Mesos-Master machine. My SPARK_HOME = /vol1/spark/spark-1.0.2-bin-hadoop2
I have uploaded the spark-1.0.2-bin-hadoop2.tgz to hdfs so that the mesos slaves can download it to invoke the Mesos Spark backend executor. But on submitting the job, I can see the below error in 'stderr' logs on the Mesos slave machine: *sh: /vol1/spark/spark-1.0.2-bin-hadoop2/sbin/spark-executor: No such file or directory* Based on documentation,I understand that if I keep the spark-mesos binary file in hdfs,I dont need to install Spark separately on the slave nodes.So, the SPARK_HOME or /vol1/spark/spark-1.0.2-bin-hadoop2/ path is non-existent on any of my slave machines and hence the error. Now, my questions is: Shouldn't the mesos-slave be looking for the spark-executor command in the temporary directory where it is supposed to extract the spark-1.0.2-bin-hadoop2.tgz from hdfs,instead of the SPARK_HOME directory?What am I doing wrong here? Any help would be really appreciated. Thanks, Bijoy
