Did you run start-slave.sh (not start-slaves.sh) in the worker machine directly? If so, is it a permission issue that it cannot run the compute-classpath.sh?
TD On Fri, Feb 14, 2014 at 10:34 AM, Guanhua Yan <gh...@lanl.gov> wrote: > Thanks TD. Yeah, I understand that. The reason for doing so is that we > have a lot of desktops, which do not form a commodity cluster but can be > used together to accomplish a common computational task. > > Following your suggestion yesterday, I again ran into trouble. On the > slave machine, it got the following exception: > > java.io.IOException: Cannot run program > "/home1/ghyan/Software/spark-0.9.0-incubating-bin-hadoop2/bin/compute-classpath.sh" > (in directory "."): error=2, No such file or directory > > The path "/home1/ghyan/Software/spark-0.9.0-incubating-bin-hadoop2" > actually exists on the master machine. Obviously it is passed to the slave > machine. > > Best regards, > - Guanhua > > > > From: Tathagata Das <tathagata.das1...@gmail.com> > Reply-To: <user@spark.incubator.apache.org> > Date: Thu, 13 Feb 2014 17:39:53 -0800 > > To: <user@spark.incubator.apache.org> > Subject: Re: Cluster launch > > I am not entirely sure if that was the intended configuration for the > scripts, but that is probably how it currently is since the most common > configuration involves the same SPARK_HOME on all machines. > > TD > > > On Thu, Feb 13, 2014 at 1:53 PM, Guanhua Yan <gh...@lanl.gov> wrote: > >> Thanks, TD. It seems that in order to use the magic start-*.sh to launch >> a cluster, all the nodes should have the same SPARK_HOME setting. >> >> Best regards, >> - Guanhua >> >> From: Tathagata Das <tathagata.das1...@gmail.com> >> Reply-To: <user@spark.incubator.apache.org> >> Date: Thu, 13 Feb 2014 13:12:21 -0800 >> To: <user@spark.incubator.apache.org> >> Subject: Re: Cluster launch >> >> You could use sbin/start-slave.sh on the slave machine to launch the >> slave. That should use the local SPARK_HOME on the slave machine to launch >> the worker correctly. >> >> TD >> >> >> On Thu, Feb 13, 2014 at 1:09 PM, Guanhua Yan <gh...@lanl.gov> wrote: >> >>> Hi all: >>> >>> I was trying to run sbin/start-master.sh and sbin/start-slaves.sh for >>> launching a standalone cluster, which contains a linux workstation and a >>> mac desktop. On these two computers, the SPARK_HOME directories point to >>> different places. When running ./sbin/start-slaves.sh, I got an error >>> saying that on the slave machine, the spark directory doesn't exist. I >>> guess that in the start-slaves.sh script, the SPARK_HOME configuration on >>> the master machine was used when launching spark on the slave machine. >>> >>> Any clues about how to fix this? >>> >>> Thank you, >>> - Guanhua >>> >>> >> >
