you can't do this with the contrib/ec2 scripts/ami.
but passing the master private dns name to the slaves on boot as 'user-
data' works fine. when a slave starts, it contacts the master and
joins the cluster. there isn't any need for a slave to rsync from the
master, thus removing the dependency on them having the private key.
and not using the start|stop-all scripts, you don't need to maintain
the slaves file, and can thus lazily boot your cluster.
to do this, you will need to create your own AMI that works this way.
not hard, just time consuming.
On Mar 20, 2008, at 11:56 AM, Prasan Ary wrote:
Chris,
What do you mean when you say boot the slaves with "the master
private name" ?
=======================
Chris K Wensel <[EMAIL PROTECTED]> wrote:
I found it much better to start the master first, then boot the
slaves
with the master private name.
i do not use the start|stop-all scrips, so i do not need to maintain
the slaves file. thus i don't need to push private keys around to
support those scripts.
this lets me start 20 nodes, then add 20 more later. or kill some.
btw, get ganglia installed. life will be better knowing what's going
on.
also, setting up FoxyProxy on firefox lets you browse your whole
cluster if you setup a ssh tunnel (socks).
On Mar 20, 2008, at 10:15 AM, Prasan Ary wrote:
Hi All,
I have been trying to configure Hadoop on EC2 for large number of
clusters ( 100 plus). It seems that I have to copy EC2 private key
to all the machines in the cluster so that they can have SSH
connections.
For now it seems I have to run a script to copy the key file to
each of the EC2 instances. I wanted to know if there is a better way
to accomplish this.
Thanks,
PA
---------------------------------
Never miss a thing. Make Yahoo your homepage.
Chris K Wensel
[EMAIL PROTECTED]
http://chris.wensel.net/
---------------------------------
Looking for last minute shopping deals? Find them fast with Yahoo!
Search.
Chris K Wensel
[EMAIL PROTECTED]
http://chris.wensel.net/
