Aaron Kimball wrote:
Cam,
This isn't Hadoop-specific, it's how Linux treats its network configuration.
If you look at /etc/host.conf, you'll probably see a line that says "order
hosts, bind" -- this is telling Linux's DNS resolution library to first read
your /etc/hosts file, then check an external DNS server.
You could probably disable local hostfile checking, but that means that
every time a program on your system queries the authoritative hostname for
"localhost", it'll go out to the network. You'll probably see a big
performance hit. The better solution, I think, is to get your nodes'
/etc/hosts files squared away.
I agree
You only need to do so once :)
No, you need to detect whenever the Linux networking stack has decided
to add new entries to resolv.conf or /etc/hosts and detect when they
are inappropriate. Which is a tricky thing to do as there are some cases
where you may actually be grateful that someone in the debian codebase
decided that adding the local hostname as 127.0.0.1 is actually a
feature. I ended up writing a new SmartFrog component that can be
configured to fail to start if the network is a mess, which is something
worth pushing out.
as part of hadoop diagnostics, this test would be one of the things to
deal with and at least warn on. "your hostname is local, you will not be
visible over the network".
-steve
