On Wed, Aug 26, 2009 at 1:32 PM, MilleBii<[email protected]> wrote: > beware you could create a kind of Denial of Service attack if you search the > site too quickly. >
Well, since I fixed Nutch so that it understands "Last-Modified" and "If-Modified-Since", it won't be downloading the pages 99% of the time, just getting a "304" response. -- http://www.linkedin.com/in/paultomblin
