On Fri, May 15, 2009 at 07:09:58AM -0700, Simon Yeo wrote: > Has anyone see this error in haproxy logs? > > Somewhere between 65k and 70k sessions, haproxy starts spewing the > error and things start dying from that point. CPU(s) get pegged at > 100%, healthchecks start failing, etc. > > Some relevant information: > -- incoming bandwith is no more than 50Mbps at 65k sessions > -- Centos 5.2 (kernel 2.6.18-92.1.22.el5) > -- haproxy-1.3.14.11 > > Config details: > maxconn 400000 > mode http > balance roundrobin > option httpchk HEAD / HTTP/1.0 > option forwardfor except 127.0.0.1 > option httpclose > each server gets "check inter 3s source [ip-address]", ip-address varies
This happens because your system does not know any spare source port to bind to in order to connect to a server. My question is : do you really have *that* many connections ? The largest number of connections I have seen on real traffic was about 30000 on a very large download site (long connections). More commonly, large sites rarely go beyond 3-5000. If you were just running a benchmark, I'm less worried of course. If you're really experiencing production issues, I'd like to get a bit more information about the type of traffic, number of servers, number of configuration sections, etc... In fact, I already have a per-IP source port allocator in another project that I should not have too many difficulties getting merged in haproxy. The problem I have is what to attach the pools to (backend, servers, IP, ...), reason why I'm interested in your environment and configuration. Regards, Willy
