Hi folks
Been using haproxy for a while now and love it load balancing apache and nginx
web server clusters and I am glad to have stumbled across this forum :)
The question I have is, is it possible to rate limit spider and bots by user
agent from haproxy level ? i.e. rate limit yandex and bai
Hey,
Hi,
For now, you can only track users by IP.
cheers
On Mon, Feb 13, 2012 at 10:14 PM, wrote:
> Hi folks
>
> Been using haproxy for a while now and love it load balancing apache and
> nginx web server clusters and I am glad to have stumbled across this forum :)
>
> The question I have
o:hapr...@serverphorums.com]
> Sent: Monday, February 13, 2012 4:15 PM
> To: haproxy@formilux.org
> Subject: Rate limit spider / bots ?
>
> Hi folks
>
>
>
> Been using haproxy for a while now and love it load balancing apache and
> nginx web server clusters and I am gla
John how would i go about using acl ? I thought rate-limit option didn't
support backends http://code.google.com/p/haproxy-docs/wiki/rate_limit_sessions
?
---
posted at http://www.serverphorums.com
http://www.serverphorums.com/read.php?10,445347,446647#msg-446647
esday, February 15, 2012 9:39 PM
> To: haproxy@formilux.org
> Subject: Re: Rate limit spider / bots ?
>
> John how would i go about using acl ? I thought rate-limit option didn't
> support backends http://code.google.com/p/haproxy-
> docs/wiki/rate_limit_sessions ?
>
Thanks John gives me other ideas would this work
###
frontend www_fe
bind :80
mode http
maxconn 4096
default_backend www_be
option contstats
acl spiderbots hdr_sub(user-agent) -i -f /etc/haproxy/spiderbotlist.lst
use_backend spider_backend if spiderbots
backend spider_backend
acl too_fast fe_se
Hi,
Let me just update a bit your configuration:
frontend www_fe
bind :8080
mode http
maxconn 4096
default_backend www_be
acl spiderbots hdr_sub(user-agent) -i -f /etc/haproxy/spiderbotlist.lst
use_backend spider_backend if spiderbots
backend www_be
mode http
server be1 127.0.0.
sweet so bumping tcp-request inspect-delay from 1500ms to 3s dramatically
slowed down activity :)
tried apachebench with tcp-request inspect-delay of 1.5s, 2s and 3s giving me
217 req/s, 97 req/s, and 67 req/s respectively. Without rate limiting around
948 req/s
thanks Baptiste
---
posted at
you're welcome :)
I've added an article on our blog: http://blog.exceliance.fr/ about
this piece of configuration which is easy to implement and is quite
efficient :)
cheers
On Fri, Feb 17, 2012 at 6:00 PM, wrote:
> sweet so bumping tcp-request inspect-delay from 1500ms to 3s dramatically
> s
yeah very simple implementation
i have it working on a live site right now - pretty good to see live spider bot
activity on haproxy admin stats end too
if you blog posted config, what does this live do exactly
acl spiderbots hdr_cnt(User-Agent) eq 0
is it to because bots head length usually z
I just did that to provide some exampes.
This week, I was working on a customer Aloha (Exceliance LB)
installation when I saw that some spiders were browsing the website.
And they had no useragent header. So I decided to add this kind of
example in my configuration.
Main purpose is to show people
11 matches
Mail list logo
