On 1/24/26 11:12, Constantine A. Murenin wrote:
On Fri, 23 Jan 2026 at 16:37, Страхиња Радић <[email protected] <mailto:[email protected]>> wrote:
...
It also happens when the browser doesn't supply the referrer header.https://marc.info/?t=176402576800001&r=1&w=2 <https://marc.info/?t=176402576800001&r=1&w=2> https://marc.info/?l=openbsd-misc&m=176406809623993&w=2 <https://marc.info/?l=openbsd-misc&m=176406809623993&w=2> https://marc.info/?l=openbsd-misc&m=176408086402658&w=2 <https://marc.info/?l=openbsd-misc&m=176408086402658&w=2> tl;dr: It's an anti-spam/anti-scraping measure. There's no way this is even effective; because the "AI" bots simply send out the fake www.google.com <http://www.google.com> and other popular websites in place of the referral, which is only worse, because now they're spamming your analytics, too.
It is very effective, in that cvsweb is running now. Before Bob's anti-attack systems were put into place, cvsweb had an expected uptime of a few MINUTES before scrapers overwhelmed the resources of the system. Now we are going maintenance update to maintenance update with just a rare wedge and self-fix. Ten years ago, cvsweb ran as one function of a Pentium-D with 3G of RAM, and never broke a sweat unless someone's robots.txt file handling was broke (this did happen at least once, and when notified, the offender offered their apologies and fixed it). Now it's running by itself on a six core monster with 48GB RAM...and being swamped by these damned web scrapers who ignore robots.txt and try to get every possible permutation of diff from every single file in CVS...as fast as possible.
It's also horrible for usability.
I wish I could argue with that, but I can't...other than to say, It beats being shut down. I HAVE changed the redirection to localhost. I am looking at ideas for a "better" solution, which I'm sure will be hated by many, because it isn't as straight forward as it was. I don't even know what the better solution is, I just know it won't be liked, and I won't like doing it. Nick.
