On Wed, Jul 16, 2025 at 11:57 AM Chris Adams via NANOG <[email protected]> wrote: > The problem with a lot of the "AI" scrapers is that they're apparently > using botnets and will often only make a single request from a given IP > address, so reactive blocking doesn't work
So do reactive un-blocking instead. The first request from any IP address gets an empty web page with, "<meta http-equiv=refresh content=1>," and triggers a permit rule which allows subsequent requests to reach the actual web server and content. That way the one-shot sources get an empty page no matter what URL they request. Regards, Bill Herrin -- William Herrin [email protected] https://bill.herrin.us/ _______________________________________________ NANOG mailing list https://lists.nanog.org/archives/list/[email protected]/message/IDU4FU7HZVRHUZWJ6SLQ7UH64JQ4IYD3/
