My test setup is three Dell r410 servers (dual Intel(R) Xeon(R) CPU X5650 @
2.67GHz - 24 threads total, 128Gb RAM) all connected to 1Gbps network.
One server is haproxy, configured to block all requests with
'Accept-Encoding: none':
global
daemon
maxconn 80000
option forwardfor
retries 10
frontend public
bind 192.168.0.1:80
default_backend nginx
acl accepts_none hdr(Accept-Encoding) -i none
errorfile 403 /raid/emptypage.txt
block if accepts_none
backend nginx
server srv 127.0.0.1:80 maxconn 8192
File /raid/emptypage.txt is an empty file made with 'touch
/raid/emptypage.txt'.
I'm doing ab2 -c 1000 -H 'Accept-Encoding: None' -n 1000000
http://192.168.0.1/ on two other servers and get following:
When nbproc = 1 haproxy saturates 100% of cpu core it runs at, but server is
running nice, I'm able to get reply from nginx behind by using curl on my
machine: curl http://192.168.0.1/, ab reports 16833 requests/second each and
longest request is around 14seconds.
When I change nbproc to higher values (maximum is 24 as there 24 threads
total) I can see ksoftirq/0 process saturating cpu core, network becomes
slow on server, ab reports same 16k-17k requests/second for each client, but
longest request is always around 20-30 seconds.
I've seen such things with ksoftirq/0 running at 100% and network is almost
down during DDoS attacks in case of too many iptables rules but what is
happening now? And what number to use at nbproc? Is it ok to have haproxy
running at 100%? It looks like I can have 30k requests per second in my
setup, is there any way to make it higher? I've done some basic tuning like
tcp_max_tw_buckets = 1024*1024, tcp_tw_reuse = 1, tcp_max_syn_backlog =
30000. Am I running out of options?
