Heh. Perhaps we should eliminate the default user-agent string? Then
he'd have less of a target to aim at... :)
On a more serious note, it seems reasonable to require a customized
"bot" URL at least. But publishing an email contact is questionable
these days. Neither Y! nor G do it, precisely because it will just
get spammed. (Wait until a spam-bot crawls this blogspot page and
starts hammering nutch-agent...)
On Jun 14, 2006, at 1:03 PM, Doug Cutting wrote:
http://incredibill.blogspot.com/2006/06/how-much-nutch-is-too-much-
nutch.html
--
Matt Kangas / [EMAIL PROTECTED]
