Thank you Magnus for your heroic efforts. It sounds like a frustrating task.
And yet, isn't there a simple solution? 1. Provide a read/write interface to GitLab for logged-in authenticated users only 2. Also provide a read-only Gitlab, feeding off the same database (or conceivably a regularly updated snapshot thereof) for unauthenticated users and bots. Bots, scrapers, and scammers are not (I hope) authenticated users, so they'd be stuck with (2) which might be slow. Meanwhile all of us authenticated users would be using the lightly loaded (1) server. Would that not be a simple solution? But I'm not an infrastructure guy, so I'm probably just blowing hot air. Simon On Thu, 6 Aug 2026 at 17:58, Magnus Viernickel via ghc-devs < [email protected]> wrote: > Dear GHC Devs, > > I have some interesting information to share with you. > You might have noticed that our gitlab was a bit more swamped the recent > days again and today I investigated and discovered that Anubis was > getting bypassed by requests with a new user agent that I haven't seen > before, "Lightpanda/1.0". > > Normally, anubis checks for user agents that are known bot user agents > and blocks them; Lightpanda was left unblocked because Anubis only > filtered for LightPanda (capital p). > > This is normally not a problem because most crawlers try to use user > agents that look like browsers; if you're using a browser user agent, > you're getting a higher weight and you get presented as challenge; but > you're also much harder to sift out because you look benign. > > While the amount of requests we were getting with that user agent had > reached absurd dimensions, this was also a blessing. First of all, it > was easy to ban it and reduce the gitlab load drastically. > > Second, now we could run an analysis on who is using these malicious > clients. Xe, the creator of Anubis kindly provided me with a tool that > qualitatively (based on unique IP addresses, disregarding hits per IP) > puts IP addresses in buckets based on several known factors, e.g. > location or the ASN (which entity the IP address block is delegated to). > > The results can be seen here: https://gitlab.haskell.org/-/snippets/6132 > > A few noteworthy things: > - 1.4 Million distinct IPv4 addresses have been used with this bot user > agent > - the IP addresses stem from 225 different countries. I think we have ~ > 30 more distinct ISO country codes. > - we have some known faces! like spacex which got flagged 100% time on > their 21000 distinct IP addresses they're using to DDoS us > - we also have quite a lot of domestic use ASNs, which indicates that > botnets are involved, too! > - Xe noted, that we can proudly say this counts as a DDoS > > Mind that this does not include all the spammers that are actually > trying to hide the fact that they're spammers, which still exist. Also > mind that this was just the data from ~1 day of logs. > > I hope this gives a nice sense of what we're dealing with and maybe it > also helps with your frustration to see my frustration and the fact that > we're sometimes getting little wins. Like today. > > Best > > Magnus > > _______________________________________________ > ghc-devs mailing list -- [email protected] > To unsubscribe send an email to [email protected] >
_______________________________________________ ghc-devs mailing list -- [email protected] To unsubscribe send an email to [email protected]
