So I know I can use Haproxy to send 429s when a given request rate is exceeded.
I have a case where the "user" is mostly screen scrapers and click bots, so if I return a 429 they'll just turn around and re-request until successful - I can't expect them to voluntarily manage their request rate or do any sort of back-off when requests fail. So instead I want to keep the connections open and the requests alive, and just delay dispatching them to an upstream backend. Is there anyway I can do something like this? I'm open to suggestions of alternative ways to achieve the same effect.
