+Andrew Actually I think this is because we haven't uploaded the Spark binaries to cloudfront / pushed the change to mesos/spark-ec2.
Andrew, can you take care of this ? On Tue, Dec 2, 2014 at 5:11 PM, Nicholas Chammas <nicholas.cham...@gmail.com > wrote: > Interesting. Do you have any problems when launching in us-east-1? What > is the full output of spark-ec2 when launching a cluster? (Post it to a > gist if it’s too big for email.) > > > On Mon, Dec 1, 2014 at 10:34 AM, Dave Challis <dave.chal...@aistemos.com> > wrote: > >> I've been trying to create a Spark cluster on EC2 using the >> documentation at https://spark.apache.org/docs/latest/ec2-scripts.html >> (with Spark 1.1.1). >> >> Running the script successfully creates some EC2 instances, HDFS etc., >> but appears to fail to copy the actual files needed to run Spark >> across. >> >> I ran the following commands: >> >> $ cd ~/src/spark-1.1.1/ec2 >> $ ./spark-ec2 --key-pair=* --identity-file=* --slaves=1 >> --region=eu-west-1 --zone=eu-west-1a --instance-type=m3.medium >> --no-ganglia launch foocluster >> >> I see the following in the script's output: >> >> (instance and HDFS set up happens here) >> ... >> Persistent HDFS installed, won't start by default... >> ~/spark-ec2 ~/spark-ec2 >> Setting up spark-standalone >> RSYNC'ing /root/spark/conf to slaves... >> *****.eu-west-1.compute.amazonaws.com >> RSYNC'ing /root/spark-ec2 to slaves... >> *****.eu-west-1.compute.amazonaws.com >> ./spark-standalone/setup.sh: line 22: /root/spark/sbin/stop-all.sh: No >> such file or directory >> ./spark-standalone/setup.sh: line 27: >> /root/spark/sbin/start-master.sh: No such file or directory >> ./spark-standalone/setup.sh: line 33: >> /root/spark/sbin/start-slaves.sh: No such file or directory >> Setting up tachyon >> RSYNC'ing /root/tachyon to slaves... >> ... >> (Tachyon setup happens here without any problem) >> >> I can ssh to the master (using the ./spark-ec2 login), and looking in >> /root/, it contains: >> >> $ ls /root >> ephemeral-hdfs hadoop-native mapreduce persistent-hdfs scala >> shark spark spark-ec2 tachyon >> >> If I look in /root/spark (where the sbin directory should be found), >> it only contains a single 'conf' directory: >> >> $ ls /root/spark >> conf >> >> Any idea why spark-ec2 might have failed to copy these files across? >> >> Thanks, >> Dave >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >> For additional commands, e-mail: user-h...@spark.apache.org >> >> >
