On Fri, Apr 20, 2012 at 06:38:41AM +0200, Baptiste wrote: > On Thu, Apr 19, 2012 at 3:48 PM, Joel Svensson > <joel__svens...@hotmail.com> wrote: > > Hi! > > > > From the text below I can't figure out if HAProxy will handle more sessions > > (than ~20000/GB ram) in Layer4 mode? > > > > The session concurrency > > This factor is tied to the previous one. Generally, the session rate will > > drop when the number of concurrent sessions increases (except the epoll > > polling mechanism). The slower the servers, the higher the number of > > concurrent sessions for a same session rate. If a load balancer receives > > 10000 sessions per second and the servers respond in 100 ms, then the load > > balancer will have 1000 concurrent sessions. This number is limited by the > > amount of memory and the amount of file-descriptors the system can handle. > > With 8 kB buffers, HAProxy will need about 16 kB per session, which results > > in around 60000 sessions per GB of RAM. In practise, socket buffers in the > > system also need some memory and 20000 sessions per GB of RAM is more > > reasonable. Layer 4 load balancers generally announce millions of > > simultaneous sessions because they don't process any data so they don't need > > any buffer. Moreover, they are sometimes designed to be used in Direct > > Server Return mode, in which the load balancer only sees forward traffic, > > and which forces it to keep the sessions for a long time after their end to > > avoid cutting sessions before they are closed. > > > > > > Regards, > > Joel > > > > > Hi Joel, > > I guess that by layer4 mode, you mean tcp mode. > Unfortunately, when Willy speaks about layer 4 load-baancers in the > text above, he speaks about LVS like load-balancers
Exactly. The difference is between LBs that process a stream and which are proxy-based, and the ones which process packets and are basically routers. In order to parse and modify a stream, you need some memory, while you don't need this to route packets (beyond the routing queue). L4 load balancers often store a session table which is a few hundreds of bytes per session, as opposed to a few tens of kB of buffers for proxies. However, L4 LBs have to deal with TIME_WAIT, which proxies don't since it's done in the system, so in practice, the ratio is not really tens-of-thousands to millions but rather tens-of-thousands to hundreds-of-thousands when the connection rate are high. > and why in HAProxy > you can have "only" thousends of connections while LVS like LBs can > annouce millions... > So in haproxy, whatever the mode, tcp or http, you'll always have > thousends of connexions. In fact it depends a lot on the configured memory and on the kernel tuning. With todays 64-bit systems and cheap RAM, there's plenty of margin. We had one user who reported 1 million established connections in a bench, and several ones reported more than 300k in production. In Linux, by default, processes are limited to 1 million FDs so you need to patch the kernel or to run in multi-process mode for this. I assume it's not that crazy to run several processes when you have to deal with 1 million concurrent connections :-) Cheers, Willy
