RE: spark 1.2 ec2 launch script hang

2015-01-28 Thread ey-chih chow
We found the problem and already fixed it. Basically, spark-ec2 requires ec2 instances to have external ip addresses. You need to specify this in the ASW console. From: nicholas.cham...@gmail.com Date: Tue, 27 Jan 2015 17:19:21 + Subject: Re: spark 1.2 ec2 launch script hang

Re: spark 1.2 ec2 launch script hang

2015-01-28 Thread Nicholas Chammas
fixed it. Basically, spark-ec2 requires ec2 instances to have external ip addresses. You need to specify this in the ASW console. -- From: nicholas.cham...@gmail.com Date: Tue, 27 Jan 2015 17:19:21 + Subject: Re: spark 1.2 ec2 launch script hang To: charles.fed

Re: spark 1.2 ec2 launch script hang

2015-01-28 Thread Charles Feduke
addresses. You need to specify this in the ASW console. -- From: nicholas.cham...@gmail.com Date: Tue, 27 Jan 2015 17:19:21 + Subject: Re: spark 1.2 ec2 launch script hang To: charles.fed...@gmail.com; pzybr...@gmail.com; eyc...@hotmail.com CC: user

Re: spark 1.2 ec2 launch script hang

2015-01-28 Thread Charles Feduke
...@gmail.com Date: Tue, 27 Jan 2015 17:19:21 + Subject: Re: spark 1.2 ec2 launch script hang To: charles.fed...@gmail.com; pzybr...@gmail.com; eyc...@hotmail.com CC: user@spark.apache.org For those who found that absolute vs. relative path for the pem file mattered, what OS and shell

Re: spark 1.2 ec2 launch script hang

2015-01-28 Thread Nicholas Chammas
instances to have external ip addresses. You need to specify this in the ASW console. -- From: nicholas.cham...@gmail.com Date: Tue, 27 Jan 2015 17:19:21 + Subject: Re: spark 1.2 ec2 launch script hang To: charles.fed...@gmail.com; pzybr...@gmail.com; eyc

Re: spark 1.2 ec2 launch script hang

2015-01-28 Thread Nicholas Chammas
addresses. You need to specify this in the ASW console. -- From: nicholas.cham...@gmail.com Date: Tue, 27 Jan 2015 17:19:21 + Subject: Re: spark 1.2 ec2 launch script hang To: charles.fed...@gmail.com; pzybr...@gmail.com; eyc...@hotmail.com CC: user

Re: spark 1.2 ec2 launch script hang

2015-01-28 Thread Peter Zybrick
to specify this in the ASW console. -- From: nicholas.cham...@gmail.com Date: Tue, 27 Jan 2015 17:19:21 + Subject: Re: spark 1.2 ec2 launch script hang To: charles.fed...@gmail.com; pzybr...@gmail.com; eyc...@hotmail.com CC: user@spark.apache.org For those who

Re: spark 1.2 ec2 launch script hang

2015-01-27 Thread Charles Feduke
Absolute path means no ~ and also verify that you have the path to the file correct. For some reason the Python code does not validate that the file exists and will hang (this is the same reason why ~ hangs). On Mon, Jan 26, 2015 at 10:08 PM Pete Zybrick pzybr...@gmail.com wrote: Try using an

Re: spark 1.2 ec2 launch script hang

2015-01-27 Thread Nicholas Chammas
For those who found that absolute vs. relative path for the pem file mattered, what OS and shell are you using? What version of Spark are you using? ~/ vs. absolute path shouldn’t matter. Your shell will expand the ~/ to the absolute path before sending it to spark-ec2. (i.e. tilde expansion.)

Re: spark 1.2 ec2 launch script hang

2015-01-26 Thread Pete Zybrick
Try using an absolute path to the pem file On Jan 26, 2015, at 8:57 PM, ey-chih chow eyc...@hotmail.com wrote: Hi, I used the spark-ec2 script of spark 1.2 to launch a cluster. I have modified the script according to

spark 1.2 ec2 launch script hang

2015-01-26 Thread ey-chih chow
Hi, I used the spark-ec2 script of spark 1.2 to launch a cluster. I have modified the script according to https://github.com/grzegorz-dubicki/spark/commit/5dd8458d2ab9753aae939b3bb33be953e2c13a70 But the script was still hung at the following message: Waiting for cluster to enter 'ssh-ready'