Hi Corin,
On Fri, Aug 19, 2022 at 08:55:17AM +0000, Corin Langosch wrote:
> Hello guys,
>
> I'm using the docker image 2.5.7-2ef551d with basic rate limiting configured
> like this:
>
> backend test
> acl test_rate_limit_by_ip_exceeds_limit
> src,table_http_req_rate(test_rate_limit_by_ip) gt 50000
> http-request deny deny_status 429 if test_rate_limit_by_ip_exceeds_limit
> http-request track-sc0 src table test_rate_limit_by_ip
>
> acl test_rate_limit_api_exceeds_limit
> req.hdr(authorization),table_http_req_rate(test_rate_limit_api) gt 100
> http-request deny deny_status 429 if test_rate_limit_api_exceeds_limit
> http-request track-sc1 req.hdr(authorization) table test_rate_limit_api
> if { path -i -m beg "/api" }
>
> acl test_rate_limit_graphql_exceeds_limit
> req.hdr(x-api-key),table_http_req_rate(test_rate_limit_graphql) gt 100
> http-request deny deny_status 429 if test_rate_limit_graphql_exceeds_limit
> http-request track-sc2 req.hdr(authorization) table
> test_rate_limit_graphql if { path -i -m beg "/graphql" }
>
> server s1 10.0.0.1:80
> server s2 10.0.0.2:80
>
> backend test_rate_limit_by_ip
> stick-table type ipv6 size 1m expire 24h store http_req_rate(5m)
>
> backend test_rate_limit_api
> stick-table type string size 1m expire 24h store http_req_rate(3m)
>
> backend test_rate_limit_graphql
> stick-table type string size 1m expire 24h store http_req_rate(3m)
>
> Now I have some users reporting they are blocked (getting a 429) even though
> they don't perform a lot of requests. To analyze I ran "show table
> table_rate_limit_by_ip", but the output looks a bit weierd to me. Here's an
> anonymized extract:
>
> 0x55cfd58df130: key=::ffff:1.2.3.1 use=0 exp=1498762521
> http_req_rate(300000)=1181926
> 0x55cfd627f840: key=::ffff:1.2.3.2 use=0 exp=1599966154
> http_req_rate(300000)=80740
> 0x55cfda287e00: key=::ffff:1.2.3.3 use=0 exp=58273431
> http_req_rate(300000)=0
> 0x55cfd5cd5320: key=::ffff:1.2.3.4 use=0 exp=2006327751
> http_req_rate(300000)=1606589
>
> I wonder what's the value of exp? It doesn't seem to be a unix timestamp (not
> even in milliseconds), nor does it seem to be the number of (milli)seconds
> till the entry expires. Unfortunately, I can't find any documentation about
> it.
For me it was the remaining time before expiration in milliseconds, and
I've just rechecked the code to be sure, it's this. It obviously doesn't
fit with your tables. Your tables have 24h expirations and these ones are
up to 23 days.
> The http_req_rate (which should be the rate within the last 5 minutes for
> this table) is extremely (unrealistic) high for many entries. How is this
> possible?
It makes me think that something is wrong with the date, though I can't
imagine what. Is it entirely reproducible or does it happen spuriously ?
I'm wondering if it could be a memory corruption for example. Could you
please share the output of "show info" ? (beware there's the host name
there that you might want to delete if it's sensitive to you). Maybe
we'll find something weird there.
> Is my configuration broken? Is it a bug? ...?
Just thinking, would it be possible that you first started with a very
high expiration value (by accident) and that you then adjusted the
config and reloaded ? From what I remember, the time left is preserved
in exchanges between peers, so having, say, a 24 days value, then a
reconfig to 24 hours and a reload would preserve the 24 days one
(arguably we should enforce the new limit), but that's definitely
something that would make sense to me.
> Thanks for any help,
> Corin
>
> PS: Is there any way to filter the output of show table
> table_rate_limit_by_ip by key (by ip in my case)? In the docs, I only find
> how to filter by value (by http_req_rate in my case).
Yes, there's "key" (look at the end of the line):
show table <name> [ data.<type> <operator> <value> [data.<type> ...]] | [ key
<key> ]
So you should be able to issue:
show table test_rate_limit_by_ip key ::ffff:1.2.3.1
And it should work.
Willy
