Hi all, I'm running Nutch to crawl some specific websites that I know the web admins personally. So is there anyway to change the settings of the target web servers such that they give my Nutch higher priority, let's say unlimited access, assuming they are all Apache servers? Because usually I observed that Nutch has a lot of "HTTP max delay" even when I set the timeout quite large and the network connections are quite perfect (I also double check by visiting those websites by browser, and they respond well).
All suggestions are welcomed. Thanks a lot. Best regards, Giang
