On 16/04/16 01:53, Jim Freeman wrote:
I'm suspecting that a connection to the stats port goes wonky with a
'-sf' reload, but I'll have to wait for it to re-appear to poke
further. I'll look first for a stats port connection handled by the
pegged process, then use 'tcpkill' to kill just that connection
(rather than the whole process, which may be handling other
connections).
We use haproxy 1.6.3 (latest CentOS 6.7) and experience similar
situation after some reloads (-sf). The old haproxy process does not
exit and uses 100% cpu, strace showing:
epoll_wait(0, {}, 200, 0) = 0
epoll_wait(0, {}, 200, 0) = 0
epoll_wait(0, {}, 200, 0) = 0
epoll_wait(0, {}, 200, 0) = 0
epoll_wait(0, {}, 200, 0) = 0
epoll_wait(0, {}, 200, 0) = 0
In our case, it was a tcp backend tunnelling rsyslog messages. After
restarting local rsyslogd, the load was gone and old haproxy instance
exited. It's hard to tell how many reloads it takes to make haproxy go
crazy or what is the exact reproducible test. But it does not take
hundreds of restart, rather 10-20 (our reloads are not very frequent) to
make haproxy go crazy.
$ haproxy -vv
HA-Proxy version 1.6.3 2015/12/25
Copyright 2000-2015 Willy Tarreau <wi...@haproxy.org>
Build options :
TARGET = linux2628
CPU = generic
CC = gcc
CFLAGS = -m64 -march=x86-64 -O2 -g -fno-strict-aliasing
-Wdeclaration-after-statement
OPTIONS = USE_ZLIB=1 USE_OPENSSL=1 USE_PCRE=1
Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
Encrypted password support via crypt(3): yes
Built with zlib version : 1.2.3
Compression algorithms supported : identity("identity"),
deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013
Running on OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports prefer-server-ciphers : yes
Built with PCRE version : 7.8 2008-09-05
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Built without Lua support
Built with transparent proxy support using: IP_TRANSPARENT
IPV6_TRANSPARENT IP_FREEBIND
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
Best regards,
Veiko
