"MOESSBAUER, Felix" <[email protected]> wrote Tue, 5 Nov 2024 09:09:21 +0000:
> Further, the rate-limits should be precisely documented so clients / > caching proxies can adapt to this. The limits also need to match the > retry-after header in the 429 responses. Currently s.d.o responds with > retry-after 5 (seconds), which is by far to short to overcome the > limit. Thanks for reporting this. I've been planning on documenting the rate limiting somewhere once it seems to behave reasonably well. It was added in a bit of a haste last weekend. One thing I would hesitate to put in a document somewhere for a human to read once and then base their implementation on is numbers that are likely to change. What do you think about the server setting X-RateLimit-Remaining in responses? The Retry-After has been fixed in [this patchset][] which I will try to get merged in a day or two. [this patchset]: https://salsa.debian.org/linus/dsa-puppet/-/compare/production...linus%2Fsnapshot-ratelimit?from_project_id=2990 > If rate-limiting would be implemented correctly, downstream caches > could properly cache the results and clients like apt could behave > nicely. I further recommend to use WAY higher request limits in > combination with a moving average limit on the amount of transferred > data. By that, the cheap "is my cache still valid" requests could pass, > while the more heavy payload transfers are avoided. Also clients could > hit s.d.o without reduced transfer rates, hence reducing the amount of > open handles on the server. What do you think would be a reasonable request limit? We're currently limiting number of requests per time unit since that's what killed one of the servers last week. I haven't looked into how to limit the amount of transferred data yet.
