We've recently upgraded to HAProxy 1.8.3, which we run with `nbthread 4`
(we used to run nbproc 4 with older releases). This has generally been
good, especially for stick tables & stats.

We terminate SSL and proxy a large number of long-running TCP
connections (websockets). When configuration changes (usually a server
going up or down), a reload occurs. For many hours, there may be 2-5
active HAProxy instances as the old ones drain. We use hard-stop-after
to keep it reasonable.

I have noticed that load average gets *very* high while these processes
are still present. Our web tier is usually at a load average of 5 with
16 cores. Across the board, load averages go up if stale HAProxy
instances are active. I saw as high as 34 with one instance that had 5
instances, and 100% CPU, most of it sys. Even with just 2, the loadavg
is double what it is with 1. Terminating the old process immediately
brings the load down.

Is there a regression in the 1.8 series with SO_REUSEPORT and nbthread
(we didn't see this before with nbproc) or somewhere we should start
looking? We make (relatively) heavy use of stick tables for DDoS
protection purposes and terminate SSL, but aside from that our
configuration is pretty vanilla. Nothing changed from 1.7 to 1.8 except
changing nbproc to nbthread.

Thanks!


Reply via email to