Re: Zeroconf for hadoop

2009-01-27 Thread Steve Loughran
Edward Capriolo wrote: Zeroconf is more focused on simplicity then security. One of the original problems that may have been fixes is that any program can announce any service. IE my laptop can announce that it is the DNS for google.com etc. -1 to zeroconf as it is way too chatty. Every DNS lo

Re: Zeroconf for hadoop

2009-01-26 Thread Vadim Zaliva
On Mon, Jan 26, 2009 at 11:22, Edward Capriolo wrote: > Zeroconf is more focused on simplicity then security. One of the > original problems that may have been fixes is that any program can > announce any service. IE my laptop can announce that it is the DNS for > google.com etc. I see two distin

Re: Zeroconf for hadoop

2009-01-26 Thread Raghu Angadi
nitesh bhatia wrote: Hi Apple provides opensource discovery service called Bonjour (zeroconf). Is it possible to integrate Zeroconf with Hadoop so that discovery of nodes become automatic ? Presently for setting up multi-node cluster we need to add IPs manually. Integrating it with bonjour can ma

Re: Zeroconf for hadoop

2009-01-26 Thread Edward Capriolo
Zeroconf is more focused on simplicity then security. One of the original problems that may have been fixes is that any program can announce any service. IE my laptop can announce that it is the DNS for google.com etc. I want to mention a related topic to the list. People are approaching the auto-

Re: Zeroconf for hadoop

2009-01-26 Thread nitesh bhatia
For a closed uniform system (yahoo, google), this can work best. This can provide plug-n-play type of system. Through this we can change clusters to dynamic grids. But I am not sure of outcome so far, I am reading the documentation. --nitesh On Mon, Jan 26, 2009 at 1:59 PM, Allen Wittenauer wro

Re: Zeroconf for hadoop

2009-01-26 Thread Raghu Angadi
Nitay wrote: Why not use the distributed coordination service ZooKeeper? When nodes come up they write some ephemeral file in a known ZooKeeper directory and anyone who's interested, i.e. NameNode, can put a watch on the directory and get notified when new children come up. NameNode does not do

Re: Zeroconf for hadoop

2009-01-26 Thread Nitay
Why not use the distributed coordination service ZooKeeper? When nodes come up they write some ephemeral file in a known ZooKeeper directory and anyone who's interested, i.e. NameNode, can put a watch on the directory and get notified when new children come up. On Mon, Jan 26, 2009 at 10:59 AM, Al

Re: Zeroconf for hadoop

2009-01-26 Thread Allen Wittenauer
On 1/25/09 8:45 AM, "nitesh bhatia" wrote: > Apple provides opensource discovery service called Bonjour (zeroconf). Is it > possible to integrate Zeroconf with Hadoop so that discovery of nodes become > automatic ? Presently for setting up multi-node cluster we need to add IPs > manually. Integ

Zeroconf for hadoop

2009-01-25 Thread nitesh bhatia
Hi Apple provides opensource discovery service called Bonjour (zeroconf). Is it possible to integrate Zeroconf with Hadoop so that discovery of nodes become automatic ? Presently for setting up multi-node cluster we need to add IPs manually. Integrating it with bonjour can make this process automat