On Sat, May 12, 2012 at 01:23:17PM +0200, Baptiste wrote:
> On Sat, May 12, 2012 at 1:01 PM, Bar Ziony <[email protected]> wrote:
> > Willy,
> >
> > Thank you, I will follow up with your suggestions soon.
> >
> > But I just had a production down-time with the haproxy machine:
> > After posting something to our Facebook wall (it happened twice, yesterday
> > and 3 days ago), which usually brings more traffic (but not more than we can
> > usually handle (for example before haproxy was deployed), the haproxy
> > machine got into swap, all the memory was taken (1GB) and the machine kept
> > keepalived bouncing to the backup machine (I believe because it was so
> > unresponsive).
> >
> > How can I check that further? Should I just increase the machine's RAM?
> >
> > Thanks,
> > Bar.
> >o
>
> Maybe you can share with us some sysctl (mainly the ones related to
> TCP buffers), as well as your HAProxy configuration (hiding private
> information)
> Are there any other processes which may eat memory on the machine?
tcp_mem is often quite sensible, you need to limit it if you don't have
enough RAM. You can also reduce haproxy's buffers size. I run all machines
at slightly less than 8kB which is more than enough and holds 5.5 TCP
segments, limiting copies in the kernel:
global
tune.bufsize 8030
tune.maxrewrite 1030
The numbers I'm used to see with this settings are 1/3 of the RAM used
by haproxy, 1/3 used by socket buffers in the kernel and the last 1/3
for the rest of the system.
With such numbers, you have each connection take around 17kB on haproxy,
which theorically allows up to around 40k concurrent conns on a 1GB machine.
Warning, it's very tricky to reach 40k conns per GB. Better stay safe and
aim at 20k per GB.
Regards,
Willy
