Sorry, I can't help with this issue, but if you are interested in a simple
way to launch a Spark cluster on Amazon, Spark is now offered as an
application in Amazon EMR. With this you can have a full cluster with a
few clicks:
https://aws.amazon.com/blogs/aws/new-apache-spark-on-amazon-emr/
- Arun
On Tue, Jul 7, 2015 at 4:34 PM, Pagliari, Roberto rpagli...@appcomsci.com
wrote:
I'm following the tutorial about Apache Spark on EC2. The output is the
following:
$ ./spark-ec2 -i ../spark.pem -k spark --copy launch spark-training
Setting up security groups...
Searching for existing cluster spark-training...
Latest Spark AMI: ami-19474270
Launching instances...
Launched 5 slaves in us-east-1d, regid = r-59a0d4b6
Launched master in us-east-1d, regid = r-9ba2d674
Waiting for instances to start up...
Waiting 120 more seconds...
Copying SSH key ../spark.pem to master...
ssh: connect to host ec2-54-152-15-165.compute-1.amazonaws.com port
22: Connection refused
Error connecting to host Command 'ssh -t -o StrictHostKeyChecking=no
-i ../spark.pem r...@ec2-54-152-15-165.compute-1.amazonaws.com 'mkdir -p
~/.ssh'' returned non-zero exit status 255, sleeping 30
ssh: connect to host ec2-54-152-15-165.compute-1.amazonaws.com port
22: Connection refused
Error connecting to host Command 'ssh -t -o StrictHostKeyChecking=no
-i ../spark.pem r...@ec2-54-152-15-165.compute-1.amazonaws.com 'mkdir -p
~/.ssh'' returned non-zero exit status 255, sleeping 30
ssh: Could not resolve hostname
ec2-54-152-15-165.compute-1.amazonaws.com: Name or service not known
Error connecting to host Command 'ssh -t -o StrictHostKeyChecking=no
-i ../spark.pem r...@ec2-54-152-15-165.compute-1.amazonaws.com 'mkdir -p
~/.ssh'' returned non-zero exit status 255, sleeping 30
ssh: connect to host ec2-54-152-15-165.compute-1.amazonaws.com port
22: Connection refused
Traceback (most recent call last):
File ./spark_ec2.py, line 925, in module
main()
File ./spark_ec2.py, line 766, in main
setup_cluster(conn, master_nodes, slave_nodes, zoo_nodes, opts,
True)
File ./spark_ec2.py, line 406, in setup_cluster
ssh(master, opts, 'mkdir -p ~/.ssh')
File ./spark_ec2.py, line 712, in ssh
raise e
subprocess.CalledProcessError: Command 'ssh -t -o
StrictHostKeyChecking=no -i ../spark.pem
r...@ec2-54-152-15-165.compute-1.amazonaws.com 'mkdir -p ~/.ssh''
returned non-zero exit status 255
However, I can see the six instances created on my EC2 console, and I
could even get the name of the master. I'm not sure how to fix the ssh
issue (my region is US EST).
