Re: Better understanding of nbproc vs distributing interrupts for cpu load management

2014-11-30 Thread Pavlos Parissis
On 28/11/2014 05:19 μμ, Lukas Tribus wrote: > Hi, > > >> you're right. >> If you need to scale *a lot* your SSL processing capacity in HAProxy, >> you must use multiple processes. >> That said, multiproc model has some counter parts (stats, server >> status, health checks are local to each proces

Re: Better understanding of nbproc vs distributing interrupts for cpu load management

2014-11-30 Thread Pavlos Parissis
On 28/11/2014 01:19 μμ, Baptiste wrote: > On Wed, Nov 26, 2014 at 9:48 PM, Pavlos Parissis > wrote: >> On 25/11/2014 07:08 μμ, Lukas Tribus wrote: Hi, > > Thanks for your reply. We have tried this approach and while >>> it gives > some benefit, the haproxy process itself > remains cpu-bound,

Re: Better understanding of nbproc vs distributing interrupts for cpu load management

2014-11-28 Thread Willy Tarreau
Hi Lukas, On Fri, Nov 28, 2014 at 05:19:31PM +0100, Lukas Tribus wrote: > I hope that one day we can move the SSL handshake to dedicated threads, > completely eliminating the event loop blocking and allowing a single > process to forward all the traffic while some parallel threads do all > the hea

RE: Better understanding of nbproc vs distributing interrupts for cpu load management

2014-11-28 Thread Lukas Tribus
Hi, > you're right. > If you need to scale *a lot* your SSL processing capacity in HAProxy, > you must use multiple processes. > That said, multiproc model has some counter parts (stats, server > status, health checks are local to each process, stick-tables can't be > synchronized, etc..). > With

Re: Better understanding of nbproc vs distributing interrupts for cpu load management

2014-11-28 Thread Baptiste
On Wed, Nov 26, 2014 at 9:48 PM, Pavlos Parissis wrote: > On 25/11/2014 07:08 μμ, Lukas Tribus wrote: >>> Hi, > > Thanks for your reply. We have tried this approach and while >> it gives > some benefit, the haproxy process itself > remains cpu-bound, >> with no idle time at all - with both pidstat

Re: Better understanding of nbproc vs distributing interrupts for cpu load management

2014-11-26 Thread Pavlos Parissis
On 25/11/2014 07:08 μμ, Lukas Tribus wrote: >> Hi, > > Thanks for your reply. We have tried this approach and while > it gives > some benefit, the haproxy process itself > remains cpu-bound, > with no idle time at all - with both pidstat and perf > reporting that > it uses close to 100% > of availa

RE: Better understanding of nbproc vs distributing interrupts for cpu load management

2014-11-26 Thread Lukas Tribus
Hi Chris, > No, there's one regexp on a rarely-used backend. > > Here's our (redacted) config: > https://gist.github.com/chrisotherwise/cfdd04216d622437f5fe > > Here's the output from haproxy -vv: > https://gist.github.com/chrisotherwise/23067a39fd59449d52f1 > > Here's the output from sar when we

Re: Better understanding of nbproc vs distributing interrupts for cpu load management

2014-11-26 Thread Willy Tarreau
Hi Chris, On Wed, Nov 26, 2014 at 01:10:21PM +, Chris Allen wrote: > >Does your config use a lot of regex ? This can cost a lot. Also if this > >is the case, there's a bug in 1.5.8 which magnifies the time spent > >processing regex when they're loaded from files (fixed in 1.5.9). Could > >you

Re: Better understanding of nbproc vs distributing interrupts for cpu load management

2014-11-26 Thread Chris Allen
On 26/11/2014 00:19, Willy Tarreau wrote: On Tue, Nov 25, 2014 at 09:33:30PM +, Chris Allen wrote: On 25/11/2014 18:08, Lukas Tribus wrote: I think SSL/TLS termination is the only use case where HAProxy saturates a CPU core of a current generation 3,4Ghz+ CPU, which is why scaling SSL/TLS

Re: Better understanding of nbproc vs distributing interrupts for cpu load management

2014-11-25 Thread Willy Tarreau
On Tue, Nov 25, 2014 at 09:33:30PM +, Chris Allen wrote: > > On 25/11/2014 18:08, Lukas Tribus wrote: > > I think SSL/TLS termination is the only use case where HAProxy > >saturates a CPU core of a current generation 3,4Ghz+ CPU, which is why > >scaling SSL/TLS is more complex, requiring nbp

Re: Better understanding of nbproc vs distributing interrupts for cpu load management

2014-11-25 Thread Willy Tarreau
Hi Vincent, On Tue, Nov 25, 2014 at 05:15:09PM +0100, Vincent Bernat wrote: > ??? 25 novembre 2014 16:46 +0100, Emeric Brun  : > > > Indeed on haproxy.com appliances, we made the choice to dedicate several > > cores/CPUs for NICs interrupts (manual bind, no usage of irqbalance) and > > to dedica

Re: Better understanding of nbproc vs distributing interrupts for cpu load management

2014-11-25 Thread Chris Allen
On 25/11/2014 18:08, Lukas Tribus wrote: I think SSL/TLS termination is the only use case where HAProxy saturates a CPU core of a current generation 3,4Ghz+ CPU, which is why scaling SSL/TLS is more complex, requiring nbproc> 1. Lukas Ok that's strange then because we don't have a very co

RE: Better understanding of nbproc vs distributing interrupts for cpu load management

2014-11-25 Thread Lukas Tribus
> Hi, > > Thanks for your reply. We have tried this approach and while it gives > some benefit, the haproxy process itself > remains cpu-bound, with no idle time at all - with both pidstat and perf > reporting that it uses close to 100% > of available cpu while running. I think SSL/TLS terminat

Re: Better understanding of nbproc vs distributing interrupts for cpu load management

2014-11-25 Thread Vincent Bernat
❦ 25 novembre 2014 16:46 +0100, Emeric Brun  : > Indeed on haproxy.com appliances, we made the choice to dedicate several > cores/CPUs for NICs interrupts (manual bind, no usage of irqbalance) and > to dedicate one other for the haproxy process with nbproc=1. On a related topic, did you get a ch

Re: Better understanding of nbproc vs distributing interrupts for cpu load management

2014-11-25 Thread Chris Allen
On 25/11/2014 15:46, Emeric Brun wrote: On 11/24/2014 05:46 PM, Chris Allen wrote: We have a load balancer running haproxy 1.5.8. At times of heavy load we're getting dangerously close to running out of CPU. I'd be really grateful for some definitive opinions on the relative merits of the two p

Re: Better understanding of nbproc vs distributing interrupts for cpu load management

2014-11-25 Thread Emeric Brun
On 11/24/2014 05:46 PM, Chris Allen wrote: We have a load balancer running haproxy 1.5.8. At times of heavy load we're getting dangerously close to running out of CPU. I'd be really grateful for some definitive opinions on the relative merits of the two possible solutions detailed below - as I'm

Re: Better understanding of nbproc vs distributing interrupts for cpu load management

2014-11-25 Thread Malcolm Turnbull
Chris, At loadbalancer.org we did some fairly heavy load testing of haproxy on various bits of hardware and get excellent results with a combination of using irqbalance + iptables NOTRACK. We tend to keep nbproc=1 unless that CPU is totally overloaded. Its not the most optimal solution but it is

Better understanding of nbproc vs distributing interrupts for cpu load management

2014-11-24 Thread Chris Allen
We have a load balancer running haproxy 1.5.8. At times of heavy load we're getting dangerously close to running out of CPU. I'd be really grateful for some definitive opinions on the relative merits of the two possible solutions detailed below - as I'm having trouble finding detailed and consis