Hi! I found out that in some situations under high rate of incoming connections haproxy=2.0.1 starts leaking sockets. It looks like haproxy doesn't close connections to its backends after request is finished (FIN received from client) thus leaving its server-sockets in close-wait state.
As an example this simple config starts leaking right from the start for me. And everything ends with messages: "local0.emerg Proxy server.local:18400 reached process FD limit (maxsock=4130). Please check 'ulimit-n' and restart." My config look very similar to this: global daemon uid 120 gid 126 stats socket /var/run/haproxy.sock mode 700 level admin expose-fd listeners master-worker log 127.0.0.1:516 local0 warning maxconn 2000 tune.ssl.default-dh-param 2048 defaults log global maxconn 4096 mode tcp retries 3 timeout client 1h timeout connect 5s timeout server 1h option redispatch option dontlognull listen server.local:18400 bind ipv6@::1:18400 tfo bind ipv4@127.0.0.1:18400 tfo mode tcp balance leastconn timeout server 24h timeout client 24h option dontlog-normal log 127.0.0.1:516 local1 info option httpchk GET /check HTTP/1.1\r\nHost:\ server.local http-check send-state http-check expect status 200 tcp-request inspect-delay 10s tcp-request content reject if { nbsrv lt 1 } default-server weight 50 server backend-server.local:17995 backend-server.local:17995 check port 17994 # haproxy -vvv HA-Proxy version 2.0.1-1 2019/06/27 - https://haproxy.org/ Build options : TARGET = linux-glibc CPU = generic CC = gcc CFLAGS = -O2 -g -O2 -fPIE -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fno-strict-aliasing -Wdeclaration-after-statement -fwrapv -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-old-style-declaration -Wno-ignored-qualifiers -Wno-clobbered -Wno-missing-field-initializers -Wtype-limits OPTIONS = USE_PCRE=1 USE_PCRE_JIT=1 USE_REGPARM=1 USE_GETADDRINFO=1 USE_OPENSSL=1 USE_LUA=1 USE_ZLIB=1 USE_TFO=1 USE_NS=1 Feature list : +EPOLL -KQUEUE -MY_EPOLL -MY_SPLICE +NETFILTER +PCRE +PCRE_JIT -PCRE2 -PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD -PTHREAD_PSHARED +REGPARM -STATIC_PCRE -STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT +CRYPT_H -VSYSCALL +GETADDRINFO +OPENSSL +LUA +FUTEX +ACCEPT4 -MY_ACCEPT4 +ZLIB -SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL -SYSTEMD -OBSOLETE_LINKER +PRCTL +THREAD_DUMP -EVPORTS Default settings : bufsize = 16384, maxrewrite = 1024, maxpollevents = 200 Built with multi-threading support (MAX_THREADS=64, default=32). Built with OpenSSL version : OpenSSL 1.0.1f 6 Jan 2014 Running on OpenSSL version : OpenSSL 1.0.1f 6 Jan 2014 OpenSSL library supports TLS extensions : yes OpenSSL library supports SNI : yes OpenSSL library supports : SSLv3 TLSv1.0 TLSv1.1 TLSv1.2 Built with Lua version : Lua 5.3.1 Built with network namespace support. Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND Built with zlib version : 1.2.8 Running on zlib version : 1.2.8 Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip") Built with PCRE version : 8.31 2012-07-06 Running on PCRE version : 8.31 2012-07-06 PCRE library supports JIT : no (libpcre build without JIT?) Encrypted password support via crypt(3): yes Built with the Prometheus exporter as a service Available polling systems : epoll : pref=300, test result OK poll : pref=200, test result OK select : pref=150, test result OK Total: 3 (3 usable), will use epoll. Available multiplexer protocols : (protocols marked as <default> cannot be specified using 'proto' keyword) h2 : mode=HTX side=FE|BE mux=H2 h2 : mode=HTTP side=FE mux=H2 <default> : mode=HTX side=FE|BE mux=H1 <default> : mode=TCP|HTTP side=FE|BE mux=PASS Available services : prometheus-exporter Available filters : [SPOE] spoe [COMP] compression [CACHE] cache [TRACE] trace -- Best regards, Maksim Kupriyanov