RES: RES: RES: RES: RES: RES: RES: RES: RES: High CPU Usage (HaProxy)

2013-11-23 Thread Fred Pedrisa
OK. The last point could slightly help in reducing the number of calls to kqueue and aggregate more events at once. But FreeBSD's kqueue is really fast so that should not change much. You really need to be able to pin the processes to certain CPUs, as well as the interrupts. Unfortunately I

RES: RES: RES: RES: RES: RES: RES: RES: High CPU Usage (HaProxy)

2013-11-22 Thread Fred Pedrisa
] Enviada em: terça-feira, 5 de novembro de 2013 13:33 Para: 'Willy Tarreau' Cc: 'Lukas Tribus'; 'haproxy@formilux.org' Assunto: RES: RES: RES: RES: RES: RES: RES: RES: High CPU Usage (HaProxy) OK. Do you know if you have a single or multiple interrupts on your NICs, and if they're delivered to a single

Re: RES: RES: RES: RES: RES: RES: RES: High CPU Usage (HaProxy)

2013-11-05 Thread Willy Tarreau
Hello Fred, [ first, please avoid top-posting, this is very cumbersome for replying in context afterwards, and tends to pollute subscribers mailboxes with overly large emails ] Also, can you confirm that this is a real machine and that we're not troubleshooting a VM ? Yes, this is a

Re: RES: RES: RES: RES: RES: RES: RES: High CPU Usage (HaProxy)

2013-11-05 Thread Jonathan Matthews
On 5 November 2013 11:16, Willy Tarreau w...@1wt.eu wrote: It is a Xeon E5-2650 Dual (So we have 16 physical cores to use here and 32 threads). OK. Do you know if you have a single or multiple interrupts on your NICs, and if they're delivered to a single core, multiple cores, or floating

RES: RES: RES: RES: RES: RES: RES: RES: High CPU Usage (HaProxy)

2013-11-05 Thread Fred Pedrisa
OK. Do you know if you have a single or multiple interrupts on your NICs, and if they're delivered to a single core, multiple cores, or floating around more or less randomly ? This is managed by FreeBSD, it currently have multiple queues and irq balance with msix. It seems that your numbers

Re: RES: RES: RES: RES: RES: RES: RES: RES: High CPU Usage (HaProxy)

2013-11-05 Thread Dmitry Sivachenko
On 05 нояб. 2013 г., at 19:33, Fred Pedrisa fredhp...@hotmail.com wrote: However, in FreeBSD we can't do that IRQ Assigning, like we can on linux. (As far I know). JFYI: you can assign IRQs to CPUs via cpuset -x irq (I can’t tell you if it is “like on linux” or not though).

RES: RES: RES: RES: RES: RES: High CPU Usage (HaProxy)

2013-10-28 Thread Fred Pedrisa
: RES: High CPU Usage (HaProxy) Hi Fred, On Mon, Oct 21, 2013 at 08:41:16PM -0200, Fred Pedrisa wrote: Hello, Ok. This is the top output : 2748 root1 870 30396K 21656K CPU88 28.0H 49.17% haproxy 2726 root1 450 38588K 32128K CPU24 16 21.1H 33.79% haproxy

RES: RES: RES: RES: RES: RES: High CPU Usage (HaProxy)

2013-10-28 Thread Fred Pedrisa
unavailable' So yes, a lot of recv/send calls as you said before. -Mensagem original- De: Willy Tarreau [mailto:w...@1wt.eu] Enviada em: segunda-feira, 28 de outubro de 2013 03:37 Para: Fred Pedrisa Cc: 'Lukas Tribus'; haproxy@formilux.org Assunto: Re: RES: RES: RES: RES: RES: High CPU Usage

Re: RES: RES: RES: RES: RES: RES: High CPU Usage (HaProxy)

2013-10-28 Thread Willy Tarreau
Hello Fred, On Mon, Oct 28, 2013 at 10:02:15AM -0200, Fred Pedrisa wrote: Hello, Willy. As you said, take a look : getsockopt(0x12e,0x,0x1007,0x7fffdb94,0x7fffdb90,0x0) = 0 (0x0) sendto(302,\^D\0\^V0\0\0^z\M-L-\a\0d8\0\0...,926,0x80,NULL,0x0) = 926 (0x39e)

RES: RES: RES: RES: RES: RES: RES: High CPU Usage (HaProxy)

2013-10-28 Thread Fred Pedrisa
: RES: RES: RES: High CPU Usage (HaProxy) Hello Fred, On Mon, Oct 28, 2013 at 10:02:15AM -0200, Fred Pedrisa wrote: Hello, Willy. As you said, take a look : getsockopt(0x12e,0x,0x1007,0x7fffdb94,0x7fffdb90,0x0) = 0 (0x0) sendto(302,\^D\0\^V0\0\0^z\M-L-\a\0d8\0\0...,926,0x80

High CPU Usage (HaProxy)

2013-10-21 Thread Fred Pedrisa
Hello, I am using many haproxy instances, for separated projects. This is causing a high cpu usage, and a high load in the OS up to 12.00 and so on. The question is, using just one instance, would reduce the CPU load, or it would make no difference at all ? Also, is there a way to

Re: High CPU Usage (HaProxy)

2013-10-21 Thread Jeff Zellner
Hi Fred, I imagine that your high load is due to running many instances of HAProxy, but hard to be 100% without all the information. Load indicates that there are processes waiting to execute, so by reducing the number of HAProxy processes you should see a reduced load (but still high cpu

RES: High CPU Usage (HaProxy)

2013-10-21 Thread Fred Pedrisa
2013 14:54 Para: Fred Pedrisa Cc: haproxy@formilux.org Assunto: Re: High CPU Usage (HaProxy) Hi Fred, I imagine that your high load is due to running many instances of HAProxy, but hard to be 100% without all the information. Load indicates that there are processes waiting to execute, so

Re: High CPU Usage (HaProxy)

2013-10-21 Thread Jeff Zellner
: Fred Pedrisa Cc: haproxy@formilux.org Assunto: Re: High CPU Usage (HaProxy) Hi Fred, I imagine that your high load is due to running many instances of HAProxy, but hard to be 100% without all the information. Load indicates that there are processes waiting to execute, so by reducing

RES: High CPU Usage (HaProxy)

2013-10-21 Thread Fred Pedrisa
de 2013 14:54 Para: Fred Pedrisa Cc: haproxy@formilux.org Assunto: Re: High CPU Usage (HaProxy) Hi Fred, I imagine that your high load is due to running many instances of HAProxy, but hard to be 100% without all the information. Load indicates that there are processes waiting to execute

Re: High CPU Usage (HaProxy)

2013-10-21 Thread Jeff Zellner
] Enviada em: segunda-feira, 21 de outubro de 2013 14:59 Para: Fred Pedrisa Cc: haproxy@formilux.org Assunto: Re: High CPU Usage (HaProxy) -sf will not actually kill the old process(es) immediately. They'll end when they are no longer handling the currently connected clients they have

RES: High CPU Usage (HaProxy)

2013-10-21 Thread Fred Pedrisa
15:07 Para: Fred Pedrisa Cc: haproxy@formilux.org Assunto: Re: High CPU Usage (HaProxy) Yes you could have all different applications in one process/configuration. When you 'reload' using -sf you'd have 2 processes for a while (depends on if you have long-lived connections or not). The old

RE: High CPU Usage (HaProxy)

2013-10-21 Thread Lukas Tribus
Hi Fred, I am using many haproxy instances, for separated projects. This is causing a high cpu usage, and a high load in the OS up to 12.00 and so on. The question is, using just one instance, would reduce the CPU load, or it would make no difference at all ? There is no way we can tell

RES: High CPU Usage (HaProxy)

2013-10-21 Thread Fred Pedrisa
-feira, 21 de outubro de 2013 15:23 Para: Fred Pedrisa; haproxy@formilux.org Assunto: RE: High CPU Usage (HaProxy) Hi Fred, I am using many haproxy instances, for separated projects. This is causing a high cpu usage, and a high load in the OS up to 12.00 and so on. The question is, using just

RE: RES: High CPU Usage (HaProxy)

2013-10-21 Thread Lukas Tribus
Hi Fred, FreeBSD 9.2 - Dual Xeon E5-2650 - 32 GB RAM. Haproxy 1.4 (Latest) 30.000~35.000 concurrent connections. About 200~300 Megabit/s. In totality. Alright, but we still need to know what haproxy does in this box. Can you post your configuration and explain what it does? Also we need

RES: RES: High CPU Usage (HaProxy)

2013-10-21 Thread Fred Pedrisa
@formilux.org Assunto: RE: RES: High CPU Usage (HaProxy) Hi Fred, FreeBSD 9.2 - Dual Xeon E5-2650 - 32 GB RAM. Haproxy 1.4 (Latest) 30.000~35.000 concurrent connections. About 200~300 Megabit/s. In totality. Alright, but we still need to know what haproxy does in this box. Can you post your

RE: RES: RES: High CPU Usage (HaProxy)

2013-10-21 Thread Lukas Tribus
Hi Fred, I am using a 10 Gbps Intel 520-DA2 NIC. The cpu usage in top vary per process we have something like : Haproxy - 93% Haproxy - 85% Haproxy - 50% Haproxy - 43% Haproxy - 32% Haproxy - 20% Haproxy - 15% Haproxy - 5% Haproxy - 1% About 30-40 Processes. I am just using it as

RES: RES: RES: High CPU Usage (HaProxy)

2013-10-21 Thread Fred Pedrisa
2013 19:40 Para: Fred Pedrisa; haproxy@formilux.org Assunto: RE: RES: RES: High CPU Usage (HaProxy) Hi Fred, I am using a 10 Gbps Intel 520-DA2 NIC. The cpu usage in top vary per process we have something like : Haproxy - 93% Haproxy - 85% Haproxy - 50% Haproxy - 43% Haproxy - 32% Haproxy

RE: RES: RES: RES: High CPU Usage (HaProxy)

2013-10-21 Thread Lukas Tribus
Hi, Yes, this is why I was speaking with Jeff about this. Because I suppose that these processes have a default loop, that uses a certain amount of CPU (kQueue implementation) Its not busy polling, if thats what you are referring to. CPU usage should be low with kqueue (because its fully

RES: RES: RES: RES: High CPU Usage (HaProxy)

2013-10-21 Thread Fred Pedrisa
- De: Lukas Tribus [mailto:luky...@hotmail.com] Enviada em: segunda-feira, 21 de outubro de 2013 20:08 Para: Fred Pedrisa; haproxy@formilux.org Assunto: RE: RES: RES: RES: High CPU Usage (HaProxy) Hi, Yes, this is why I was speaking with Jeff about this. Because I suppose that these processes

RE: RES: RES: RES: RES: High CPU Usage (HaProxy)

2013-10-21 Thread Lukas Tribus
Hi, Yes, the current version (for my usage) is really stable. However, you are right, because too many processes, will create too many threads, assuming I have just 16 Physical Cores... Do you believe on a good CPU usage decrease, by switching to one process only ? I can't guarantee it,

RES: RES: RES: RES: RES: High CPU Usage (HaProxy)

2013-10-21 Thread Fred Pedrisa
12.45, sometimes going up to 16.00 +/- -Mensagem original- De: Lukas Tribus [mailto:luky...@hotmail.com] Enviada em: segunda-feira, 21 de outubro de 2013 20:39 Para: Fred Pedrisa; haproxy@formilux.org Assunto: RE: RES: RES: RES: RES: High CPU Usage (HaProxy) Hi, Yes, the current version