Re: Nutch Ignoring Robots.txt

Mon, 03 Oct 2005 10:37:16 -0700 

eGrants Help Desk wrote:

The following has apparently been ignoring our robots.txt page:
NutchCVS/0.05 (Nutch; http://www.nutch.org/docs/en/bot.html; [EMAIL PROTECTED])


What IP address are these requests coming from?

Thanks,

Doug






















					Reply via email to