Re: Spark-ec2 asks for password

2014-04-22 Thread Pierre Borckmans
We’ve been experiencing this as well, and our simple solution is to actually keep trying the ssh connection instead of just waiting: Something like this: def wait_for_ssh_connection(opts, host): u.message("Waiting for ssh connection to host {}".format(host)) connected = False while (conne

Re: Spark-ec2 asks for password

2014-04-21 Thread Mayur Rustagi
Hi We have a deployment tool from GCE that we use internally for Spark. Let me know if you want access to that. Not really clean enough to opensource though :). Regards Mayur Mayur Rustagi Ph: +1 (760) 203 3257 http://www.sigmoidanalytics.com @mayur_rustagi

Re: Spark-ec2 asks for password

2014-04-18 Thread Aureliano Buendia
Frank, Thanks for the prompt reply. Unfortunately I've been experiencing this for the past few weeks on N Virginia farm, note that the latency might also depend on the instance type. I'll try to amend the ec2 script as you suggested, but that will mean waiting even longer for the cluster to come

Re: Spark-ec2 asks for password

2014-04-18 Thread Patrick Wendell
Unfortunately - I think a lot of this is due to generally increased latency on ec2 itself. I've noticed that it's way more common than it used to be for instances to come online past the "wait" timeout in the ec2 script. On Fri, Apr 18, 2014 at 9:11 PM, FRANK AUSTIN NOTHAFT wrote: > Aureliano,

Re: Spark-ec2 asks for password

2014-04-18 Thread FRANK AUSTIN NOTHAFT
Aureliano, I've been noticing this error recently as well: ssh: connect to host ec-xx-xx-xx-xx.compute-1.amazonaws.com port 22: Connection refused Error 255 while executing remote command, retrying after 30 seconds However, this isn't an issue with the spark-ec2 scripts. After the scripts fail,

Spark-ec2 asks for password

2014-04-18 Thread Aureliano Buendia
Hi, Since 0.9.0 spark-ec2 has gone unstable. During launch it throws many errors like: ssh: connect to host ec-xx-xx-xx-xx.compute-1.amazonaws.com port 22: Connection refused Error 255 while executing remote command, retrying after 30 seconds .. and recently, it prompts for passwords!: Warning: