Hi!
First, I'd like to thank Willy and the other haproxy contributors for
bringing this wonderful piece of software into the world :)
For the last 2 years now we have been running haproxy 1.3 successfully
to load balance our frontend applications and storage services. Mainly
the requests passing through our haproxy instances are WebDAV
commands. Since there were some new sought-after features announced in
the new stable 1.4 branch, yesterday we gave it a go and upgraded to
haproxy from 1.3.22 to 1.4.1 in our production environment (simply
replaced active binary with -sf switch). After the new version was
deployed our incoming traffic slowly dropped from approximately
150Mbps to 80Mbps (as ongoing requests were still processed by
1.3.22). The configuration file were not changed between the two
versions, so we have not yet started use any of the new config options
for 1.4 (http-server-close etc). Because of the drop in throughput we
have now rolled back to 1.3.22 (and traffic levels are back to
normal).
What differ our service from most other online services is that we are
more of a "content-consumer" rather than a content provider. The
requests that are generating our traffic volume is mostly large and
small PUT requests with Transfer-Encoding: chunked. Is this type of
requests included in any of your tests or benchmarks? Do you have a
clue of what might have changed in the code base to cause this
behavior? Any suggestions for where to go from here (other than
sticking with 1.3 :) is greatly appreciated.
The hardware this haproxy setup is running on is HP DL360G5. Base OS
is Debian Lenny (currently with kernel 2.6.18). Stunnel (w/ hw
acceleration) is used for SSL-decoding.
Parts of our haproxy.cfg:
global
log loghost local0 debug
maxconn 26000
ulimit-n 65536
user haproxy
group haproxy
chroot /var/chroot/haproxy
daemon
pidfile /var/run/haproxy.pid
spread-checks 5
defaults
log global
mode http
maxconn 10000
timeout client 60s
timeout server 60s
timeout queue 60s
timeout connect 4s
timeout http-request 5s
retries 2
option httpclose
option forceclose
option redispatch
balance roundrobin
monitor-uri /haproxy_mon
option httplog
option dontlognull
frontend censored_example
bind 10.10.10.10:80
option forwardfor except 10.10.10.10/24
"some santity acls used for blocking"
"some acls for backend matching"
capture request header something
capture cookie something
use_backend a if x
use_backend b if y
default_backend c
backend a
balance uri depth 2
server server1 server1:80 weight 1 check inter 10000
server serverN serverN:80 weight 1 check inter 10000
option httpchk GET /health
and so on... :)
Output of "haproxy -vv" for 1.3.22:
HA-Proxy version 1.3.22 2009/10/14
Copyright 2000-2009 Willy Tarreau <[email protected]>
Build options :
TARGET = linux26
CPU = generic
CC = gcc
CFLAGS = -O2 -g
OPTIONS =
Default settings :
maxconn = 2000, maxpollevents = 200
Available polling systems :
sepoll : pref=400, test result OK
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 4 (4 usable), will use sepoll.
Output of "haproxy -vv" for 1.4.1:
HA-Proxy version 1.4.1 2010/03/04
Copyright 2000-2010 Willy Tarreau <[email protected]>
Build options :
TARGET = linux26
CPU = generic
CC = gcc
CFLAGS = -O2 -g
OPTIONS =
Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents = 200
Encrypted password support via crypt(3): yes
Available polling systems :
sepoll : pref=400, test result OK
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 4 (4 usable), will use sepoll.
Best regards
Erik Gulliksson
--
Erik Gulliksson, [email protected]
System Administrator, Diino AB
http://www.diino.com
