Hello Chris, Doug.
Wouldn't Round Robin DNS possibly throw a wrench into things?
The spider could spider down different pipes (ip's) to the same server
using RRDNS to load balance between 2 different backbones.
Some test data...
----------------
dnsip www.microsoft.com
207.46.19.60 207.46.20.60 207.46.18.30 207.46.199.60 207.46.20.30
207.46.225.60 207.46.198.60 207.46.199.30
java
IPTest
www.microsoft.com/207.46.19.60
java IPTest
www.microsoft.com/207.46.20.60
java IPTest
www.microsoft.com/207.46.20.30
java IPTest
www.microsoft.com/207.46.199.30
java IPTest
www.microsoft.com/207.46.18.30
public class IPTest {
public static void main(String[] args) {
try {
java.net.InetAddress addr =
java.net.InetAddress.getByName("www.microsoft.com");
System.out.println(addr.toString());
} catch (Exception e) {
}
}
}
----------
6 of 1... half dozen of the other.
Jeff
-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
Nutch-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-developers
