Hi List,
With 1.8.8 ran into this, tried latest 1.9dev snapshot seems to have the
same issue..
Running with 3 threads, a template for 8 servers, and only 2 ip's in the
dns response, neither of which is actually 'up' one responds with
'L4TOUT in 1004ms' the other with 'L4CON in 0ms' on stats page.. DNS
comes from a local unbound dns-server on the same host as the haproxy
process. after a little while (10 minutes+-) haproxy go's to 300% usage
on all the 3 spin-locks and doesn't respond anymore.
There was no actual traffic passing through this machine. I am watching
the stats page.
Running with 1 thread the issue does not seem to appear.
I think 2 threads are deadlocking each other.?. and then later the 3rd
joins the waiting game.
I've added logging between a lot of most of the lock/unlock functions..
and it seems that the 2 succeeded locks below from lines 332 and 333 in
attached logfile are the last ones where threads 1 and 0 are doing
anything..
For example added logging like this everywhere ,the thread-id, the
servername, and the function its inside of is logged with printf
statements..:
static inline void health_adjust(struct server *s, short status)
{
HA_SPIN_LOCK(SERVER_LOCK, &s->lock);
printf("tid:%u LOCKED srv: %s health_adjust\n", tid, s->id);
/* return now if observing nor health check is not enabled */
if (!s->observe || !s->check.task) {
printf("tid:%u UNLOCK srv: %s health_adjust a\n", tid, s->id);
HA_SPIN_UNLOCK(SERVER_LOCK, &s->lock);
return;
}
__health_adjust(s, status);
printf("tid:%u UNLOCK srv: %s health_adjust b\n", tid, s->id);
HA_SPIN_UNLOCK(SERVER_LOCK, &s->lock);
}
This leads to the following calls that are seemingly never unlocked the
tid: stops appearing, and actual callstacks go several functions deeper
that try to gain another lock....:
tid:1 LOCKED srv: alias_srv1 process_chk_conn
tid:0 LOCKED srv: alias_srv8 event_srv_chk_w
Then a little later the tid:2 also hangs.. probably it was time to
perform another check or dns lookup or so...
(gdb) info threads
Id Target Id Frame
1 LWP 100730 of process 15238 0x00000000004ab6bc in
snr_check_ip_callback (srv=0x8022d0400, ip=0x80288e394,
ip_family=0x7fffffffe207 "\002\224\343\210\002\b") at src/server.c:3781
2 LWP 100924 of process 15238 0x00000000004ab6b4 in
snr_check_ip_callback (srv=0x8022cb000, ip=0x80288e394,
ip_family=0x7fffdfffda97 "\002\224\343\210\002\b") at src/server.c:3781
* 3 LWP 100925 of process 15238 0x000000000051ef21 in
dns_resolve_recv (dgram=0x8022483c0) at src/dns.c:1646
More gdb 'bt full' output of all 3 threads is at the bottom of the
attached logfile.
Hope someone can try and fix this :) Config and haproxy -vv also added
below.
Thanks
PiBa-NL (Pieter)
global
maxconn 3002
log /var/run/log local0 info
stats socket /tmp/haproxy.socket level admin expose-fd listeners
uid 80
gid 80
nbproc 1
nbthread 3
hard-stop-after 15m
chroot /tmp/haproxy_chroot
daemon
tune.ssl.default-dh-param 2048
log-send-hostname haproxy-pb-test
defaults
# never fail on address resolution last,libc,none
default-server init-addr last,none
stats show-legends
userlist myuserlist
user admin insecure-password pass
listen HAProxyLocalStats
bind :80 name localstats
mode http
stats enable
stats refresh 5
stats admin if TRUE
stats uri /
stats show-legends
timeout client 5000
timeout connect 5000
timeout server 5000
mailers globalmailers
mailer pbmail 192.168.0.40:25
resolvers globalresolvers
nameserver goog 127.0.0.1:53
resolve_retries 3
timeout retry 1s
hold valid 10s
frontend alias_test
bind 1.2.3.1:21 name 1.2.3.1:21 transparent
bind 1.2.3.1:10000-10002 name 1.2.3.1:10000-10002
transparent
bind 1.2.3.2:21 name 1.2.3.2:21 transparent
bind 1.2.3.2:10000-10002 name 1.2.3.2:10000-10002
transparent
bind 1:2:3::3:21 name 1:2:3::3:21 transparent
bind 1:2:3::3:10000-10002 name 1:2:3::3:10000-10002
transparent
bind 1:2:3::4:21 name 1:2:3::4:21 transparent
bind 1:2:3::4:10000-10002 name 1:2:3::4:10000-10002
transparent
bind 1.2.3.9:21 name 1.2.3.9:21 transparent
bind 1.2.3.9:10000-10002 name 1.2.3.9:10000-10002
transparent
mode http
log global
option socket-stats
option http-keep-alive
timeout client 30000
default_backend alias_back_http_ipvANY
backend alias_back_http_ipvANY
mode http
id 134
log global
# use mailers
# level info
email-alert mailers globalmailers
email-alert level info
email-alert from hap@pfs.local
email-alert to pba@pfs.local
email-alert myhostname pfs
timeout connect 30000
timeout server 30000
retries 3
option httpchk OPTIONS /
http-response set-status 123 reason Lol if TRUE
http-response set-status 999 if TRUE
server-template alias_srv 8
dns_server_name.pfs.local:809 check inter 1000 resolvers
globalresolvers init-addr last,libc,none
HA-Proxy version 1.9-dev0-03f4ec4 2018/05/17
Copyright 2000-2017 Willy Tarreau <wi...@haproxy.org>
Build options :
TARGET = freebsd
CPU = generic
CC = cc
CFLAGS = -pipe -g -fstack-protector -fno-strict-aliasing
-fno-strict-aliasing -Wdeclaration-after-statement -fwrapv
-fno-strict-overflow -Wno-address-of-packed-member -Wno-null-dereference
-Wno-unused-label -DFREEBSD_PORTS
OPTIONS = USE_GETADDRINFO=1 USE_ZLIB=1 USE_CPU_AFFINITY=1
USE_ACCEPT4=1 USE_REGPARM=1 USE_OPENSSL=1 USE_LUA=1 USE_STATIC_PCRE=1
USE_PCRE_JIT=1
Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
Built with network namespace support.
Built with zlib version : 1.2.11
Running on zlib version : 1.2.11
Compression algorithms supported : identity("identity"),
deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with PCRE version : 8.40 2017-01-11
Running on PCRE version : 8.40 2017-01-11
PCRE library supports JIT : yes
Built with multi-threading support.
Encrypted password support via crypt(3): yes
Built with transparent proxy support using: IP_BINDANY IPV6_BINDANY
Built with Lua version : Lua 5.3.4
Built with OpenSSL version : OpenSSL 1.0.2k-freebsd 26 Jan 2017
Running on OpenSSL version : OpenSSL 1.0.2o-freebsd 27 Mar 2018
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : SSLv3 TLSv1.0 TLSv1.1 TLSv1.2
Available polling systems :
kqueue : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use kqueue.
Available filters :
[TRACE] trace
[COMP] compression
[SPOE] spoe
-----------------------------------------------------------------------
And then a second issue where 100% is used by 1 thread but the stats
page actually stays responsive and all 3 threads seem to 'work', gdb
showed all 3 threads passing through the poller..
root@freebsd11:~ # echo "show activity" | socat stdio /tmp/haproxy.socket
thread_id: 2
date_now: 1526768652.287058
loops: 3762 413423203 4570
wake_cache: 55 2171 772
wake_tasks: 1 3684 7
wake_applets: 0 0 0
wake_signal: 0 0 0
poll_exp: 56 5855 779
poll_drop: 0 219 1
poll_dead: 0 0 0
poll_skip: 0 0 0
fd_skip: 0 0 0
fd_lock: 3 6 24
fd_del: 0 0 0
conn_dead: 0 0 0
stream: 0 182 225
empty_rq: 184 413418910 769
long_rq: 0 0 0
root@freebsd11:~ # echo "show activity" | socat stdio /tmp/haproxy.socket
thread_id: 2
date_now: 1526768652.614237
loops: 3762 413714499 4574
wake_cache: 55 2181 774
wake_tasks: 1 3685 7
wake_applets: 0 0 0
wake_signal: 0 0 0
poll_exp: 56 5866 781
poll_drop: 0 220 1
poll_dead: 0 0 0
poll_skip: 0 0 0
fd_skip: 0 0 0
fd_lock: 3 7 24
fd_del: 0 0 0
conn_dead: 0 0 0
stream: 0 182 227
empty_rq: 184 413710209 771
long_rq: 0 0 0
tid:0 LOCKED srv: alias_srv7 process_chk_conn
tid:0 UNLOCK srv: alias_srv7 process_chk_conn
tid:1 LOCKED srv: alias_srv2 process_chk_conn
tid:1 UNLOCK srv: alias_srv2 process_chk_conn
tid:2 LOCKED srv: alias_srv8 process_chk_conn
tid:0 LOCKED srv: alias_srv1 process_chk_conn
tid:2 UNLOCK srv: alias_srv8 process_chk_conn
tid:2 LOCKED srv: alias_srv8 event_srv_chk_w
tid:2 UNLOCK srv: alias_srv8 event_srv_chk_w
tid:2 LOCKED srv: alias_srv8 event_srv_chk_r
tid:0 UNLOCK srv: alias_srv1 process_chk_conn
tid:2 UNLOCK srv: alias_srv8 event_srv_chk_r
tid:2 LOCKED srv: alias_srv8 event_srv_chk_w
tid:2 UNLOCK srv: alias_srv8 event_srv_chk_w
tid:2 LOCKED srv: alias_srv8 wake_srv_chk
tid:2 UNLOCK srv: alias_srv8 wake_srv_chk
tid:2 LOCKED srv: alias_srv8 process_chk_conn
tid:2 UNLOCK srv: alias_srv8 process_chk_conn
tid:2 LOCKED srv: alias_srv3 process_chk_conn
tid:2 UNLOCK srv: alias_srv3 process_chk_conn
tid:2 LOCKED srv: alias_srv4 process_chk_conn
tid:2 UNLOCK srv: alias_srv4 process_chk_conn
tid:2 LOCKED srv: alias_srv5 process_chk_conn
tid:2 UNLOCK srv: alias_srv5 process_chk_conn
tid:2 LOCKED srv: alias_srv6 process_chk_conn
tid:2 UNLOCK srv: alias_srv6 process_chk_conn
tid:1 LOCKED srv: alias_srv7 process_chk_conn
tid:1 UNLOCK srv: alias_srv7 process_chk_conn
tid:2 LOCKED srv: alias_srv2 process_chk_conn
tid:2 UNLOCK srv: alias_srv2 process_chk_conn
tid:1 LOCKED srv: alias_srv8 process_chk_conn
tid:1 UNLOCK srv: alias_srv8 process_chk_conn
tid:1 LOCKED srv: alias_srv1 process_chk_conn
tid:1 UNLOCK srv: alias_srv1 process_chk_conn
tid:1 LOCKED srv: alias_srv8 event_srv_chk_w
tid:1 UNLOCK srv: alias_srv8 event_srv_chk_w
tid:1 LOCKED srv: alias_srv8 event_srv_chk_r
tid:1 UNLOCK srv: alias_srv8 event_srv_chk_r
tid:1 LOCKED srv: alias_srv1 event_srv_chk_w
tid:1 UNLOCK srv: alias_srv1 event_srv_chk_w
tid:1 LOCKED srv: alias_srv1 event_srv_chk_r
tid:1 UNLOCK srv: alias_srv1 event_srv_chk_r
tid:1 LOCKED srv: alias_srv8 event_srv_chk_w
tid:1 UNLOCK srv: alias_srv8 event_srv_chk_w
tid:1 LOCKED srv: alias_srv8 wake_srv_chk
tid:1 UNLOCK srv: alias_srv8 wake_srv_chk
tid:1 LOCKED srv: alias_srv8 process_chk_conn
tid:1 UNLOCK srv: alias_srv8 process_chk_conn
tid:2 LOCKED srv: alias_srv3 process_chk_conn
tid:2 UNLOCK srv: alias_srv3 process_chk_conn
tid:2 LOCKED srv: alias_srv4 process_chk_conn
tid:2 UNLOCK srv: alias_srv4 process_chk_conn
tid:2 LOCKED srv: alias_srv5 process_chk_conn
tid:2 UNLOCK srv: alias_srv5 process_chk_conn
tid:2 LOCKED srv: alias_srv6 process_chk_conn
tid:2 UNLOCK srv: alias_srv6 process_chk_conn
tid:2 LOCKED srv: alias_srv7 process_chk_conn
tid:2 UNLOCK srv: alias_srv7 process_chk_conn
tid:2 LOCKED srv: alias_srv2 process_chk_conn
tid:2 UNLOCK srv: alias_srv2 process_chk_conn
000001f5:HAProxyLocalStats.clireq[0022:ffffffff]: GET / HTTP/1.1
000001f5:HAProxyLocalStats.clihdr[0022:ffffffff]: Host: 192.168.8.93
000001f5:HAProxyLocalStats.clihdr[0022:ffffffff]: Connection: keep-alive
000001f5:HAProxyLocalStats.clihdr[0022:ffffffff]: Cache-Control: max-age=0
000001f5:HAProxyLocalStats.clihdr[0022:ffffffff]: Upgrade-Insecure-Requests: 1
000001f5:HAProxyLocalStats.clihdr[0022:ffffffff]: User-Agent: Mozilla/5.0
(Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/66.0.3359.181 Safari/537.36
000001f5:HAProxyLocalStats.clihdr[0022:ffffffff]: Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
000001f5:HAProxyLocalStats.clihdr[0022:ffffffff]: Referer: http://192.168.8.93/
000001f5:HAProxyLocalStats.clihdr[0022:ffffffff]: Accept-Encoding: gzip, deflate
000001f5:HAProxyLocalStats.clihdr[0022:ffffffff]: Accept-Language:
nl-NL,nl;q=0.9,en-US;q=0.8,en;q=0.7
000001f5:HAProxyLocalStats.srvrep[0022:ffffffff]: HTTP/1.1 200 OK
000001f5:HAProxyLocalStats.srvhdr[0022:ffffffff]: Cache-Control: no-cache
000001f5:HAProxyLocalStats.srvhdr[0022:ffffffff]: Connection: close
000001f5:HAProxyLocalStats.srvhdr[0022:ffffffff]: Content-Type: text/html
000001f5:HAProxyLocalStats.srvhdr[0022:ffffffff]: Refresh: 5
000001f5:HAProxyLocalStats.srvhdr[0022:ffffffff]: Transfer-Encoding: chunked
000001f5:HAProxyLocalStats.srvcls[0022:ffffffff]
tid:0 LOCKED srv: alias_srv8 process_chk_conn
tid:0 UNLOCK srv: alias_srv8 process_chk_conn
tid:0 LOCKED srv: alias_srv8 event_srv_chk_w
tid:0 UNLOCK srv: alias_srv8 event_srv_chk_w
tid:0 LOCKED srv: alias_srv8 event_srv_chk_r
tid:0 UNLOCK srv: alias_srv8 event_srv_chk_r
tid:0 LOCKED srv: alias_srv8 event_srv_chk_w
tid:0 UNLOCK srv: alias_srv8 event_srv_chk_w
tid:0 LOCKED srv: alias_srv8 wake_srv_chk
tid:0 UNLOCK srv: alias_srv8 wake_srv_chk
tid:0 LOCKED srv: alias_srv8 process_chk_conn
tid:0 UNLOCK srv: alias_srv8 process_chk_conn
tid:1 LOCKED srv: alias_srv1 process_chk_conn
tid:1 UNLOCK srv: alias_srv1 process_chk_conn
tid:1 LOCKED srv: alias_srv3 process_chk_conn
tid:1 UNLOCK srv: alias_srv3 process_chk_conn
tid:2 LOCKED srv: alias_srv4 process_chk_conn
tid:2 UNLOCK srv: alias_srv4 process_chk_conn
tid:2 LOCKED srv: alias_srv5 process_chk_conn
tid:2 UNLOCK srv: alias_srv5 process_chk_conn
tid:2 LOCKED srv: alias_srv6 process_chk_conn
tid:2 UNLOCK srv: alias_srv6 process_chk_conn
tid:0 LOCKED srv: alias_srv7 process_chk_conn
tid:0 UNLOCK srv: alias_srv7 process_chk_conn
tid:1 LOCKED srv: alias_srv2 process_chk_conn
tid:1 UNLOCK srv: alias_srv2 process_chk_conn
000001f6:HAProxyLocalStats.clireq[0022:ffffffff]: GET /favicon.ico HTTP/1.1
000001f6:HAProxyLocalStats.clihdr[0022:ffffffff]: Host: 192.168.8.93
000001f6:HAProxyLocalStats.clihdr[0022:ffffffff]: Connection: keep-alive
000001f6:HAProxyLocalStats.clihdr[0022:ffffffff]: User-Agent: Mozilla/5.0
(Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/66.0.3359.181 Safari/537.36
000001f6:HAProxyLocalStats.clihdr[0022:ffffffff]: Accept:
image/webp,image/apng,image/*,*/*;q=0.8
000001f6:HAProxyLocalStats.clihdr[0022:ffffffff]: Referer: http://192.168.8.93/
000001f6:HAProxyLocalStats.clihdr[0022:ffffffff]: Accept-Encoding: gzip, deflate
000001f6:HAProxyLocalStats.clihdr[0022:ffffffff]: Accept-Language:
nl-NL,nl;q=0.9,en-US;q=0.8,en;q=0.7
000001f6:HAProxyLocalStats.srvrep[0022:ffffffff]: HTTP/1.1 200 OK
000001f6:HAProxyLocalStats.srvhdr[0022:ffffffff]: Cache-Control: no-cache
000001f6:HAProxyLocalStats.srvhdr[0022:ffffffff]: Connection: close
000001f6:HAProxyLocalStats.srvhdr[0022:ffffffff]: Content-Type: text/html
000001f6:HAProxyLocalStats.srvhdr[0022:ffffffff]: Refresh: 5
000001f6:HAProxyLocalStats.srvhdr[0022:ffffffff]: Transfer-Encoding: chunked
000001f6:HAProxyLocalStats.srvcls[0022:ffffffff]
tid:0 LOCKED srv: alias_srv8 process_chk_conn
tid:0 UNLOCK srv: alias_srv8 process_chk_conn
tid:0 LOCKED srv: alias_srv1 process_chk_conn
tid:0 UNLOCK srv: alias_srv1 process_chk_conn
tid:0 LOCKED srv: alias_srv8 event_srv_chk_w
tid:0 UNLOCK srv: alias_srv8 event_srv_chk_w
tid:0 LOCKED srv: alias_srv8 event_srv_chk_r
tid:0 UNLOCK srv: alias_srv8 event_srv_chk_r
tid:0 LOCKED srv: alias_srv1 event_srv_chk_w
tid:0 UNLOCK srv: alias_srv1 event_srv_chk_w
tid:0 LOCKED srv: alias_srv1 event_srv_chk_r
tid:0 UNLOCK srv: alias_srv1 event_srv_chk_r
tid:0 LOCKED srv: alias_srv8 event_srv_chk_w
tid:0 UNLOCK srv: alias_srv8 event_srv_chk_w
tid:0 LOCKED srv: alias_srv8 wake_srv_chk
tid:0 UNLOCK srv: alias_srv8 wake_srv_chk
tid:0 LOCKED srv: alias_srv8 process_chk_conn
tid:0 UNLOCK srv: alias_srv8 process_chk_conn
tid:1 LOCKED srv: alias_srv3 process_chk_conn
tid:1 UNLOCK srv: alias_srv3 process_chk_conn
tid:2 LOCKED srv: alias_srv4 process_chk_conn
tid:2 UNLOCK srv: alias_srv4 process_chk_conn
tid:2 LOCKED srv: alias_srv5 process_chk_conn
tid:2 UNLOCK srv: alias_srv5 process_chk_conn
tid:1 LOCKED srv: alias_srv6 process_chk_conn
tid:1 UNLOCK srv: alias_srv6 process_chk_conn
tid:2 LOCKED srv: alias_srv7 process_chk_conn
tid:2 UNLOCK srv: alias_srv7 process_chk_conn
tid:2 LOCKED srv: alias_srv2 process_chk_conn
tid:2 UNLOCK srv: alias_srv2 process_chk_conn
tid:1 LOCKED srv: alias_srv8 process_chk_conn
tid:0 LOCKED srv: alias_srv1 process_chk_conn
tid:1 UNLOCK srv: alias_srv8 process_chk_conn
tid:1 LOCKED srv: alias_srv8 event_srv_chk_w
tid:1 UNLOCK srv: alias_srv8 event_srv_chk_w
tid:1 LOCKED srv: alias_srv8 event_srv_chk_r
tid:1 UNLOCK srv: alias_srv8 event_srv_chk_r
tid:0 UNLOCK srv: alias_srv1 process_chk_conn
tid:1 LOCKED srv: alias_srv8 event_srv_chk_w
tid:1 UNLOCK srv: alias_srv8 event_srv_chk_w
tid:1 LOCKED srv: alias_srv8 wake_srv_chk
tid:1 UNLOCK srv: alias_srv8 wake_srv_chk
tid:1 LOCKED srv: alias_srv8 process_chk_conn
tid:1 UNLOCK srv: alias_srv8 process_chk_conn
tid:2 LOCKED srv: alias_srv3 process_chk_conn
tid:2 UNLOCK srv: alias_srv3 process_chk_conn
tid:1 LOCKED srv: alias_srv4 process_chk_conn
tid:1 UNLOCK srv: alias_srv4 process_chk_conn
tid:0 LOCKED srv: alias_srv5 process_chk_conn
tid:0 UNLOCK srv: alias_srv5 process_chk_conn
tid:1 LOCKED srv: alias_srv6 process_chk_conn
tid:1 UNLOCK srv: alias_srv6 process_chk_conn
000001f7:HAProxyLocalStats.clireq[0022:ffffffff]: GET / HTTP/1.1
000001f7:HAProxyLocalStats.clihdr[0022:ffffffff]: Host: 192.168.8.93
000001f7:HAProxyLocalStats.clihdr[0022:ffffffff]: Connection: keep-alive
000001f7:HAProxyLocalStats.clihdr[0022:ffffffff]: Cache-Control: max-age=0
000001f7:HAProxyLocalStats.clihdr[0022:ffffffff]: Upgrade-Insecure-Requests: 1
000001f7:HAProxyLocalStats.clihdr[0022:ffffffff]: User-Agent: Mozilla/5.0
(Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/66.0.3359.181 Safari/537.36
000001f7:HAProxyLocalStats.clihdr[0022:ffffffff]: Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
000001f7:HAProxyLocalStats.clihdr[0022:ffffffff]: Referer: http://192.168.8.93/
000001f7:HAProxyLocalStats.clihdr[0022:ffffffff]: Accept-Encoding: gzip, deflate
000001f7:HAProxyLocalStats.clihdr[0022:ffffffff]: Accept-Language:
nl-NL,nl;q=0.9,en-US;q=0.8,en;q=0.7
000001f7:HAProxyLocalStats.srvrep[0022:ffffffff]: HTTP/1.1 200 OK
000001f7:HAProxyLocalStats.srvhdr[0022:ffffffff]: Cache-Control: no-cache
000001f7:HAProxyLocalStats.srvhdr[0022:ffffffff]: Connection: close
000001f7:HAProxyLocalStats.srvhdr[0022:ffffffff]: Content-Type: text/html
000001f7:HAProxyLocalStats.srvhdr[0022:ffffffff]: Refresh: 5
000001f7:HAProxyLocalStats.srvhdr[0022:ffffffff]: Transfer-Encoding: chunked
000001f7:HAProxyLocalStats.srvcls[0022:ffffffff]
tid:0 LOCKED srv: alias_srv7 process_chk_conn
tid:0 UNLOCK srv: alias_srv7 process_chk_conn
tid:1 LOCKED srv: alias_srv2 process_chk_conn
tid:1 UNLOCK srv: alias_srv2 process_chk_conn
tid:2 LOCKED srv: alias_srv8 process_chk_conn
tid:2 UNLOCK srv: alias_srv8 process_chk_conn
tid:2 LOCKED srv: alias_srv1 process_chk_conn
tid:2 UNLOCK srv: alias_srv1 process_chk_conn
tid:2 LOCKED srv: alias_srv8 event_srv_chk_w
tid:2 UNLOCK srv: alias_srv8 event_srv_chk_w
tid:2 LOCKED srv: alias_srv8 event_srv_chk_r
tid:2 UNLOCK srv: alias_srv8 event_srv_chk_r
tid:2 LOCKED srv: alias_srv1 event_srv_chk_w
tid:2 UNLOCK srv: alias_srv1 event_srv_chk_w
tid:2 LOCKED srv: alias_srv1 event_srv_chk_r
tid:2 UNLOCK srv: alias_srv1 event_srv_chk_r
tid:2 LOCKED srv: alias_srv8 event_srv_chk_w
tid:2 UNLOCK srv: alias_srv8 event_srv_chk_w
tid:2 LOCKED srv: alias_srv8 wake_srv_chk
tid:2 UNLOCK srv: alias_srv8 wake_srv_chk
tid:2 LOCKED srv: alias_srv8 process_chk_conn
tid:2 UNLOCK srv: alias_srv8 process_chk_conn
tid:2 LOCKED srv: alias_srv3 process_chk_conn
tid:2 UNLOCK srv: alias_srv3 process_chk_conn
000001f8:HAProxyLocalStats.clireq[0022:ffffffff]: GET /favicon.ico HTTP/1.1
000001f8:HAProxyLocalStats.clihdr[0022:ffffffff]: Host: 192.168.8.93
000001f8:HAProxyLocalStats.clihdr[0022:ffffffff]: Connection: keep-alive
000001f8:HAProxyLocalStats.clihdr[0022:ffffffff]: User-Agent: Mozilla/5.0
(Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/66.0.3359.181 Safari/537.36
000001f8:HAProxyLocalStats.clihdr[0022:ffffffff]: Accept:
image/webp,image/apng,image/*,*/*;q=0.8
000001f8:HAProxyLocalStats.clihdr[0022:ffffffff]: Referer: http://192.168.8.93/
000001f8:HAProxyLocalStats.clihdr[0022:ffffffff]: Accept-Encoding: gzip, deflate
000001f8:HAProxyLocalStats.clihdr[0022:ffffffff]: Accept-Language:
nl-NL,nl;q=0.9,en-US;q=0.8,en;q=0.7
000001f8:HAProxyLocalStats.srvrep[0022:ffffffff]: HTTP/1.1 200 OK
000001f8:HAProxyLocalStats.srvhdr[0022:ffffffff]: Cache-Control: no-cache
000001f8:HAProxyLocalStats.srvhdr[0022:ffffffff]: Connection: close
000001f8:HAProxyLocalStats.srvhdr[0022:ffffffff]: Content-Type: text/html
000001f8:HAProxyLocalStats.srvhdr[0022:ffffffff]: Refresh: 5
000001f8:HAProxyLocalStats.srvhdr[0022:ffffffff]: Transfer-Encoding: chunked
000001f8:HAProxyLocalStats.srvcls[0022:ffffffff]
tid:1 LOCKED srv: alias_srv4 process_chk_conn
tid:1 UNLOCK srv: alias_srv4 process_chk_conn
tid:1 LOCKED srv: alias_srv5 process_chk_conn
tid:1 UNLOCK srv: alias_srv5 process_chk_conn
tid:2 LOCKED srv: alias_srv6 process_chk_conn
tid:2 UNLOCK srv: alias_srv6 process_chk_conn
tid:1 LOCKED srv: alias_srv7 process_chk_conn
tid:1 UNLOCK srv: alias_srv7 process_chk_conn
tid:2 LOCKED srv: alias_srv2 process_chk_conn
tid:2 UNLOCK srv: alias_srv2 process_chk_conn
tid:0 LOCKED srv: alias_srv8 process_chk_conn
tid:0 UNLOCK srv: alias_srv8 process_chk_conn
tid:0 LOCKED srv: alias_srv8 event_srv_chk_w
tid:0 UNLOCK srv: alias_srv8 event_srv_chk_w
tid:0 LOCKED srv: alias_srv8 event_srv_chk_r
tid:0 UNLOCK srv: alias_srv8 event_srv_chk_r
tid:0 LOCKED srv: alias_srv8 event_srv_chk_w
tid:0 UNLOCK srv: alias_srv8 event_srv_chk_w
tid:0 LOCKED srv: alias_srv8 wake_srv_chk
tid:0 UNLOCK srv: alias_srv8 wake_srv_chk
tid:0 LOCKED srv: alias_srv8 process_chk_conn
tid:0 UNLOCK srv: alias_srv8 process_chk_conn
tid:2 LOCKED srv: alias_srv1 process_chk_conn
tid:2 UNLOCK srv: alias_srv1 process_chk_conn
tid:0 LOCKED srv: alias_srv3 process_chk_conn
tid:0 UNLOCK srv: alias_srv3 process_chk_conn
tid:2 LOCKED srv: alias_srv4 process_chk_conn
tid:2 UNLOCK srv: alias_srv4 process_chk_conn
tid:0 LOCKED srv: alias_srv5 process_chk_conn
tid:0 UNLOCK srv: alias_srv5 process_chk_conn
tid:0 LOCKED srv: alias_srv6 process_chk_conn
tid:0 UNLOCK srv: alias_srv6 process_chk_conn
tid:1 LOCKED srv: alias_srv7 process_chk_conn
tid:1 UNLOCK srv: alias_srv7 process_chk_conn
tid:2 LOCKED srv: alias_srv2 process_chk_conn
tid:2 UNLOCK srv: alias_srv2 process_chk_conn
tid:1 LOCKED srv: alias_srv8 process_chk_conn
tid:1 UNLOCK srv: alias_srv8 process_chk_conn
tid:1 LOCKED srv: alias_srv1 process_chk_conn
tid:1 UNLOCK srv: alias_srv1 process_chk_conn
tid:1 LOCKED srv: alias_srv8 event_srv_chk_w
tid:1 UNLOCK srv: alias_srv8 event_srv_chk_w
tid:1 LOCKED srv: alias_srv8 event_srv_chk_r
tid:1 UNLOCK srv: alias_srv8 event_srv_chk_r
tid:1 LOCKED srv: alias_srv1 event_srv_chk_w
tid:1 UNLOCK srv: alias_srv1 event_srv_chk_w
tid:1 LOCKED srv: alias_srv1 event_srv_chk_r
tid:1 UNLOCK srv: alias_srv1 event_srv_chk_r
tid:1 LOCKED srv: alias_srv8 event_srv_chk_w
tid:1 UNLOCK srv: alias_srv8 event_srv_chk_w
tid:1 LOCKED srv: alias_srv8 wake_srv_chk
tid:1 UNLOCK srv: alias_srv8 wake_srv_chk
tid:1 LOCKED srv: alias_srv8 process_chk_conn
tid:1 UNLOCK srv: alias_srv8 process_chk_conn
tid:0 LOCKED srv: alias_srv3 process_chk_conn
tid:0 UNLOCK srv: alias_srv3 process_chk_conn
tid:2 LOCKED srv: alias_srv1 event_srv_chk_w
tid:2 UNLOCKED srv: alias_srv1 event_srv_chk_w
tid:2 LOCKED srv: alias_srv2 event_srv_chk_w
tid:2 UNLOCKED srv: alias_srv2 event_srv_chk_w
tid:2 LOCKED srv: alias_srv3 event_srv_chk_w
tid:2 UNLOCKED srv: alias_srv3 event_srv_chk_w
tid:2 LOCKED srv: alias_srv4 event_srv_chk_w
tid:2 UNLOCKED srv: alias_srv4 event_srv_chk_w
tid:2 LOCKED srv: alias_srv5 event_srv_chk_w
tid:2 UNLOCKED srv: alias_srv5 event_srv_chk_w
tid:2 LOCKED srv: alias_srv6 event_srv_chk_w
tid:2 UNLOCKED srv: alias_srv6 event_srv_chk_w
tid:2 LOCKED srv: alias_srv7 event_srv_chk_w
tid:2 UNLOCKED srv: alias_srv7 event_srv_chk_w
tid:2 LOCKED srv: alias_srv8 event_srv_chk_w
tid:2 UNLOCKED srv: alias_srv8 event_srv_chk_w
tid:0 LOCKED srv: alias_srv4 process_chk_conn
tid:0 UNLOCK srv: alias_srv4 process_chk_conn
tid:1 LOCKED srv: alias_srv5 process_chk_conn
tid:1 UNLOCK srv: alias_srv5 process_chk_conn
tid:0 LOCKED srv: alias_srv6 process_chk_conn
tid:0 UNLOCK srv: alias_srv6 process_chk_conn
tid:0 LOCKED srv: alias_srv7 process_chk_conn
tid:0 UNLOCK srv: alias_srv7 process_chk_conn
tid:1 LOCKED srv: alias_srv2 process_chk_conn
tid:1 UNLOCK srv: alias_srv2 process_chk_conn
000001f9:HAProxyLocalStats.clireq[0022:ffffffff]: GET / HTTP/1.1
000001f9:HAProxyLocalStats.clihdr[0022:ffffffff]: Host: 192.168.8.93
000001f9:HAProxyLocalStats.clihdr[0022:ffffffff]: Connection: keep-alive
000001f9:HAProxyLocalStats.clihdr[0022:ffffffff]: Cache-Control: max-age=0
000001f9:HAProxyLocalStats.clihdr[0022:ffffffff]: Upgrade-Insecure-Requests: 1
000001f9:HAProxyLocalStats.clihdr[0022:ffffffff]: User-Agent: Mozilla/5.0
(Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/66.0.3359.181 Safari/537.36
000001f9:HAProxyLocalStats.clihdr[0022:ffffffff]: Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
000001f9:HAProxyLocalStats.clihdr[0022:ffffffff]: Referer: http://192.168.8.93/
000001f9:HAProxyLocalStats.clihdr[0022:ffffffff]: Accept-Encoding: gzip, deflate
000001f9:HAProxyLocalStats.clihdr[0022:ffffffff]: Accept-Language:
nl-NL,nl;q=0.9,en-US;q=0.8,en;q=0.7
000001f9:HAProxyLocalStats.srvrep[0022:ffffffff]: HTTP/1.1 200 OK
000001f9:HAProxyLocalStats.srvhdr[0022:ffffffff]: Cache-Control: no-cache
000001f9:HAProxyLocalStats.srvhdr[0022:ffffffff]: Connection: close
000001f9:HAProxyLocalStats.srvhdr[0022:ffffffff]: Content-Type: text/html
000001f9:HAProxyLocalStats.srvhdr[0022:ffffffff]: Refresh: 5
000001f9:HAProxyLocalStats.srvhdr[0022:ffffffff]: Transfer-Encoding: chunked
000001f9:HAProxyLocalStats.srvcls[0022:ffffffff]
tid:0 LOCKED srv: alias_srv8 process_chk_conn
tid:0 UNLOCK srv: alias_srv8 process_chk_conn
tid:0 LOCKED srv: alias_srv8 event_srv_chk_w
tid:0 UNLOCK srv: alias_srv8 event_srv_chk_w
tid:0 LOCKED srv: alias_srv8 event_srv_chk_r
tid:0 UNLOCK srv: alias_srv8 event_srv_chk_r
tid:1 LOCKED srv: alias_srv1 process_chk_conn
tid:0 LOCKED srv: alias_srv8 event_srv_chk_w
tid:2 LOCKED srv: alias_srv3 process_chk_conn
tid:2 UNLOCK srv: alias_srv3 process_chk_conn
tid:2 LOCKED srv: alias_srv4 process_chk_conn
tid:2 UNLOCK srv: alias_srv4 process_chk_conn
tid:2 LOCKED srv: alias_srv5 process_chk_conn
tid:2 UNLOCK srv: alias_srv5 process_chk_conn
tid:2 LOCKED srv: alias_srv6 process_chk_conn
tid:2 UNLOCK srv: alias_srv6 process_chk_conn
tid:2 LOCKED srv: alias_srv7 process_chk_conn
tid:2 UNLOCK srv: alias_srv7 process_chk_conn
tid:2 LOCKED srv: alias_srv2 process_chk_conn
tid:2 UNLOCK srv: alias_srv2 process_chk_conn
000001fb:HAProxyLocalStats.accept(0005)=0025 from [192.168.202.222:57381]
ALPN=<none>
000001fb:HAProxyLocalStats.clireq[0025:ffffffff]: GET /favicon.ico HTTP/1.1
000001fb:HAProxyLocalStats.clihdr[0025:ffffffff]: Host: 192.168.8.93
000001fb:HAProxyLocalStats.clihdr[0025:ffffffff]: Connection: keep-alive
000001fb:HAProxyLocalStats.clihdr[0025:ffffffff]: User-Agent: Mozilla/5.0
(Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/66.0.3359.181 Safari/537.36
000001fb:HAProxyLocalStats.clihdr[0025:ffffffff]: Accept:
image/webp,image/apng,image/*,*/*;q=0.8
000001fb:HAProxyLocalStats.clihdr[0025:ffffffff]: Referer: http://192.168.8.93/
000001fb:HAProxyLocalStats.clihdr[0025:ffffffff]: Accept-Encoding: gzip, deflate
000001fb:HAProxyLocalStats.clihdr[0025:ffffffff]: Accept-Language:
nl-NL,nl;q=0.9,en-US;q=0.8,en;q=0.7
000001fb:HAProxyLocalStats.srvrep[0025:ffffffff]: HTTP/1.1 200 OK
000001fb:HAProxyLocalStats.srvhdr[0025:ffffffff]: Cache-Control: no-cache
000001fb:HAProxyLocalStats.srvhdr[0025:ffffffff]: Connection: close
000001fb:HAProxyLocalStats.srvhdr[0025:ffffffff]: Content-Type: text/html
000001fb:HAProxyLocalStats.srvhdr[0025:ffffffff]: Refresh: 5
000001fb:HAProxyLocalStats.srvhdr[0025:ffffffff]: Transfer-Encoding: chunked
000001fb:HAProxyLocalStats.srvcls[0025:ffffffff]
tid:2 LOCKED srv: alias_srv3 process_chk_conn
tid:2 UNLOCK srv: alias_srv3 process_chk_conn
tid:2 LOCKED srv: alias_srv4 process_chk_conn
tid:2 UNLOCK srv: alias_srv4 process_chk_conn
tid:2 LOCKED srv: alias_srv5 process_chk_conn
tid:2 UNLOCK srv: alias_srv5 process_chk_conn
tid:2 LOCKED srv: alias_srv6 process_chk_conn
tid:2 UNLOCK srv: alias_srv6 process_chk_conn
tid:2 LOCKED srv: alias_srv7 process_chk_conn
tid:2 UNLOCK srv: alias_srv7 process_chk_conn
tid:2 LOCKED srv: alias_srv2 process_chk_conn
tid:2 UNLOCK srv: alias_srv2 process_chk_conn
tid:2 LOCKED srv: alias_srv3 process_chk_conn
tid:2 UNLOCK srv: alias_srv3 process_chk_conn
000001fc:HAProxyLocalStats.clireq[0025:ffffffff]: GET / HTTP/1.1
000001fc:HAProxyLocalStats.clihdr[0025:ffffffff]: Host: 192.168.8.93
000001fc:HAProxyLocalStats.clihdr[0025:ffffffff]: Connection: keep-alive
000001fc:HAProxyLocalStats.clihdr[0025:ffffffff]: Cache-Control: max-age=0
000001fc:HAProxyLocalStats.clihdr[0025:ffffffff]: Upgrade-Insecure-Requests: 1
000001fc:HAProxyLocalStats.clihdr[0025:ffffffff]: User-Agent: Mozilla/5.0
(Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/66.0.3359.181 Safari/537.36
000001fc:HAProxyLocalStats.clihdr[0025:ffffffff]: Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
000001fc:HAProxyLocalStats.clihdr[0025:ffffffff]: Referer: http://192.168.8.93/
000001fc:HAProxyLocalStats.clihdr[0025:ffffffff]: Accept-Encoding: gzip, deflate
000001fc:HAProxyLocalStats.clihdr[0025:ffffffff]: Accept-Language:
nl-NL,nl;q=0.9,en-US;q=0.8,en;q=0.7
000001fc:HAProxyLocalStats.srvrep[0025:ffffffff]: HTTP/1.1 200 OK
000001fc:HAProxyLocalStats.srvhdr[0025:ffffffff]: Cache-Control: no-cache
000001fc:HAProxyLocalStats.srvhdr[0025:ffffffff]: Connection: close
000001fc:HAProxyLocalStats.srvhdr[0025:ffffffff]: Content-Type: text/html
000001fc:HAProxyLocalStats.srvhdr[0025:ffffffff]: Refresh: 5
000001fc:HAProxyLocalStats.srvhdr[0025:ffffffff]: Transfer-Encoding: chunked
000001fc:HAProxyLocalStats.srvcls[0025:ffffffff]
tid:2 LOCKED srv: alias_srv4 process_chk_conn
tid:2 UNLOCK srv: alias_srv4 process_chk_conn
tid:2 LOCKED srv: alias_srv5 process_chk_conn
tid:2 UNLOCK srv: alias_srv5 process_chk_conn
tid:2 LOCKED srv: alias_srv6 process_chk_conn
tid:2 UNLOCK srv: alias_srv6 process_chk_conn
tid:2 LOCKED srv: alias_srv7 process_chk_conn
tid:2 UNLOCK srv: alias_srv7 process_chk_conn
000001fd:HAProxyLocalStats.clireq[0025:ffffffff]: GET /favicon.ico HTTP/1.1
000001fd:HAProxyLocalStats.clihdr[0025:ffffffff]: Host: 192.168.8.93
000001fd:HAProxyLocalStats.clihdr[0025:ffffffff]: Connection: keep-alive
000001fd:HAProxyLocalStats.clihdr[0025:ffffffff]: User-Agent: Mozilla/5.0
(Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/66.0.3359.181 Safari/537.36
000001fd:HAProxyLocalStats.clihdr[0025:ffffffff]: Accept:
image/webp,image/apng,image/*,*/*;q=0.8
000001fd:HAProxyLocalStats.clihdr[0025:ffffffff]: Referer: http://192.168.8.93/
000001fd:HAProxyLocalStats.clihdr[0025:ffffffff]: Accept-Encoding: gzip, deflate
000001fd:HAProxyLocalStats.clihdr[0025:ffffffff]: Accept-Language:
nl-NL,nl;q=0.9,en-US;q=0.8,en;q=0.7
000001fd:HAProxyLocalStats.srvrep[0025:ffffffff]: HTTP/1.1 200 OK
000001fd:HAProxyLocalStats.srvhdr[0025:ffffffff]: Cache-Control: no-cache
000001fd:HAProxyLocalStats.srvhdr[0025:ffffffff]: Connection: close
000001fd:HAProxyLocalStats.srvhdr[0025:ffffffff]: Content-Type: text/html
000001fd:HAProxyLocalStats.srvhdr[0025:ffffffff]: Refresh: 5
000001fd:HAProxyLocalStats.srvhdr[0025:ffffffff]: Transfer-Encoding: chunked
000001fd:HAProxyLocalStats.srvcls[0025:ffffffff]
tid:2 LOCKED srv: alias_srv2 process_chk_conn
tid:2 UNLOCK srv: alias_srv2 process_chk_conn
tid:2 LOCKED srv: alias_srv3 process_chk_conn
tid:2 UNLOCK srv: alias_srv3 process_chk_conn
tid:2 LOCKED srv: alias_srv4 process_chk_conn
tid:2 UNLOCK srv: alias_srv4 process_chk_conn
tid:2 LOCKED srv: alias_srv5 process_chk_conn
tid:2 UNLOCK srv: alias_srv5 process_chk_conn
tid:2 LOCKED srv: alias_srv6 process_chk_conn
tid:2 UNLOCK srv: alias_srv6 process_chk_conn
tid:2 LOCKED srv: alias_srv7 process_chk_conn
tid:2 UNLOCK srv: alias_srv7 process_chk_conn
tid:2 LOCKED srv: alias_srv2 process_chk_conn
tid:2 UNLOCK srv: alias_srv2 process_chk_conn
tid:2 LOCKED srv: alias_srv3 process_chk_conn
tid:2 UNLOCK srv: alias_srv3 process_chk_conn
tid:2 LOCKED srv: alias_srv4 process_chk_conn
tid:2 UNLOCK srv: alias_srv4 process_chk_conn
tid:2 LOCKED srv: alias_srv5 process_chk_conn
tid:2 UNLOCK srv: alias_srv5 process_chk_conn
tid:2 LOCKED srv: alias_srv6 process_chk_conn
tid:2 UNLOCK srv: alias_srv6 process_chk_conn
tid:2 LOCKED srv: alias_srv7 process_chk_conn
tid:2 UNLOCK srv: alias_srv7 process_chk_conn
tid:2 LOCKED srv: alias_srv2 process_chk_conn
tid:2 UNLOCK srv: alias_srv2 process_chk_conn
tid:2 LOCKED srv: alias_srv3 process_chk_conn
tid:2 UNLOCK srv: alias_srv3 process_chk_conn
tid:2 LOCKED srv: alias_srv4 process_chk_conn
tid:2 UNLOCK srv: alias_srv4 process_chk_conn
tid:2 LOCKED srv: alias_srv5 process_chk_conn
tid:2 UNLOCK srv: alias_srv5 process_chk_conn
tid:2 LOCKED srv: alias_srv6 process_chk_conn
tid:2 UNLOCK srv: alias_srv6 process_chk_conn
tid:2 LOCKED srv: alias_srv7 process_chk_conn
tid:2 UNLOCK srv: alias_srv7 process_chk_conn
tid:2 LOCKED srv: alias_srv2 process_chk_conn
tid:2 UNLOCK srv: alias_srv2 process_chk_conn
000001fe:HAProxyLocalStats.clireq[0025:ffffffff]: GET / HTTP/1.1
000001fe:HAProxyLocalStats.clihdr[0025:ffffffff]: Host: 192.168.8.93
000001fe:HAProxyLocalStats.clihdr[0025:ffffffff]: Connection: keep-alive
000001fe:HAProxyLocalStats.clihdr[0025:ffffffff]: Cache-Control: max-age=0
000001fe:HAProxyLocalStats.clihdr[0025:ffffffff]: Upgrade-Insecure-Requests: 1
000001fe:HAProxyLocalStats.clihdr[0025:ffffffff]: User-Agent: Mozilla/5.0
(Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/66.0.3359.181 Safari/537.36
000001fe:HAProxyLocalStats.clihdr[0025:ffffffff]: Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
000001fe:HAProxyLocalStats.clihdr[0025:ffffffff]: Referer: http://192.168.8.93/
000001fe:HAProxyLocalStats.clihdr[0025:ffffffff]: Accept-Encoding: gzip, deflate
000001fe:HAProxyLocalStats.clihdr[0025:ffffffff]: Accept-Language:
nl-NL,nl;q=0.9,en-US;q=0.8,en;q=0.7
000001fe:HAProxyLocalStats.srvrep[0025:ffffffff]: HTTP/1.1 200 OK
000001fe:HAProxyLocalStats.srvhdr[0025:ffffffff]: Cache-Control: no-cache
000001fe:HAProxyLocalStats.srvhdr[0025:ffffffff]: Connection: close
000001fe:HAProxyLocalStats.srvhdr[0025:ffffffff]: Content-Type: text/html
000001fe:HAProxyLocalStats.srvhdr[0025:ffffffff]: Refresh: 5
000001fe:HAProxyLocalStats.srvhdr[0025:ffffffff]: Transfer-Encoding: chunked
000001fe:HAProxyLocalStats.srvcls[0025:ffffffff]
tid:2 LOCKED srv: alias_srv3 process_chk_conn
tid:2 UNLOCK srv: alias_srv3 process_chk_conn
tid:2 LOCKED srv: alias_srv4 process_chk_conn
tid:2 UNLOCK srv: alias_srv4 process_chk_conn
tid:2 LOCKED srv: alias_srv5 process_chk_conn
tid:2 UNLOCK srv: alias_srv5 process_chk_conn
tid:2 LOCKED srv: alias_srv6 process_chk_conn
tid:2 UNLOCK srv: alias_srv6 process_chk_conn
tid:2 LOCKED srv: alias_srv7 process_chk_conn
tid:2 UNLOCK srv: alias_srv7 process_chk_conn
tid:2 LOCKED srv: alias_srv2 process_chk_conn
tid:2 UNLOCK srv: alias_srv2 process_chk_conn
000001ff:HAProxyLocalStats.clireq[0025:ffffffff]: GET /favicon.ico HTTP/1.1
000001ff:HAProxyLocalStats.clihdr[0025:ffffffff]: Host: 192.168.8.93
000001ff:HAProxyLocalStats.clihdr[0025:ffffffff]: Connection: keep-alive
000001ff:HAProxyLocalStats.clihdr[0025:ffffffff]: User-Agent: Mozilla/5.0
(Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/66.0.3359.181 Safari/537.36
000001ff:HAProxyLocalStats.clihdr[0025:ffffffff]: Accept:
image/webp,image/apng,image/*,*/*;q=0.8
000001ff:HAProxyLocalStats.clihdr[0025:ffffffff]: Referer: http://192.168.8.93/
000001ff:HAProxyLocalStats.clihdr[0025:ffffffff]: Accept-Encoding: gzip, deflate
000001ff:HAProxyLocalStats.clihdr[0025:ffffffff]: Accept-Language:
nl-NL,nl;q=0.9,en-US;q=0.8,en;q=0.7
000001ff:HAProxyLocalStats.srvrep[0025:ffffffff]: HTTP/1.1 200 OK
000001ff:HAProxyLocalStats.srvhdr[0025:ffffffff]: Cache-Control: no-cache
000001ff:HAProxyLocalStats.srvhdr[0025:ffffffff]: Connection: close
000001ff:HAProxyLocalStats.srvhdr[0025:ffffffff]: Content-Type: text/html
000001ff:HAProxyLocalStats.srvhdr[0025:ffffffff]: Refresh: 5
000001ff:HAProxyLocalStats.srvhdr[0025:ffffffff]: Transfer-Encoding: chunked
000001ff:HAProxyLocalStats.srvcls[0025:ffffffff]
tid:2 LOCKED srv: alias_srv3 process_chk_conn
tid:2 UNLOCK srv: alias_srv3 process_chk_conn
tid:2 LOCKED srv: alias_srv4 process_chk_conn
tid:2 UNLOCK srv: alias_srv4 process_chk_conn
tid:2 LOCKED srv: alias_srv5 process_chk_conn
tid:2 UNLOCK srv: alias_srv5 process_chk_conn
tid:2 LOCKED srv: alias_srv6 process_chk_conn
tid:2 UNLOCK srv: alias_srv6 process_chk_conn
tid:2 LOCKED srv: alias_srv7 process_chk_conn
tid:2 UNLOCK srv: alias_srv7 process_chk_conn
00000200:HAProxyLocalStats.clireq[0025:ffffffff]: GET / HTTP/1.1
00000200:HAProxyLocalStats.clihdr[0025:ffffffff]: Host: 192.168.8.93
00000200:HAProxyLocalStats.clihdr[0025:ffffffff]: Connection: keep-alive
00000200:HAProxyLocalStats.clihdr[0025:ffffffff]: Cache-Control: max-age=0
00000200:HAProxyLocalStats.clihdr[0025:ffffffff]: Upgrade-Insecure-Requests: 1
00000200:HAProxyLocalStats.clihdr[0025:ffffffff]: User-Agent: Mozilla/5.0
(Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/66.0.3359.181 Safari/537.36
00000200:HAProxyLocalStats.clihdr[0025:ffffffff]: Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
00000200:HAProxyLocalStats.clihdr[0025:ffffffff]: Referer: http://192.168.8.93/
00000200:HAProxyLocalStats.clihdr[0025:ffffffff]: Accept-Encoding: gzip, deflate
00000200:HAProxyLocalStats.clihdr[0025:ffffffff]: Accept-Language:
nl-NL,nl;q=0.9,en-US;q=0.8,en;q=0.7
00000200:HAProxyLocalStats.srvrep[0025:ffffffff]: HTTP/1.1 200 OK
00000200:HAProxyLocalStats.srvhdr[0025:ffffffff]: Cache-Control: no-cache
00000200:HAProxyLocalStats.srvhdr[0025:ffffffff]: Connection: close
00000200:HAProxyLocalStats.srvhdr[0025:ffffffff]: Content-Type: text/html
00000200:HAProxyLocalStats.srvhdr[0025:ffffffff]: Refresh: 5
00000200:HAProxyLocalStats.srvhdr[0025:ffffffff]: Transfer-Encoding: chunked
00000200:HAProxyLocalStats.srvcls[0025:ffffffff]
tid:2 LOCKED srv: alias_srv2 process_chk_conn
tid:2 UNLOCK srv: alias_srv2 process_chk_conn
tid:2 LOCKED srv: alias_srv3 process_chk_conn
tid:2 UNLOCK srv: alias_srv3 process_chk_conn
00000201:HAProxyLocalStats.clireq[0025:ffffffff]: GET /favicon.ico HTTP/1.1
00000201:HAProxyLocalStats.clihdr[0025:ffffffff]: Host: 192.168.8.93
00000201:HAProxyLocalStats.clihdr[0025:ffffffff]: Connection: keep-alive
00000201:HAProxyLocalStats.clihdr[0025:ffffffff]: User-Agent: Mozilla/5.0
(Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/66.0.3359.181 Safari/537.36
00000201:HAProxyLocalStats.clihdr[0025:ffffffff]: Accept:
image/webp,image/apng,image/*,*/*;q=0.8
00000201:HAProxyLocalStats.clihdr[0025:ffffffff]: Referer: http://192.168.8.93/
00000201:HAProxyLocalStats.clihdr[0025:ffffffff]: Accept-Encoding: gzip, deflate
00000201:HAProxyLocalStats.clihdr[0025:ffffffff]: Accept-Language:
nl-NL,nl;q=0.9,en-US;q=0.8,en;q=0.7
00000201:HAProxyLocalStats.srvrep[0025:ffffffff]: HTTP/1.1 200 OK
00000201:HAProxyLocalStats.srvhdr[0025:ffffffff]: Cache-Control: no-cache
00000201:HAProxyLocalStats.srvhdr[0025:ffffffff]: Connection: close
00000201:HAProxyLocalStats.srvhdr[0025:ffffffff]: Content-Type: text/html
00000201:HAProxyLocalStats.srvhdr[0025:ffffffff]: Refresh: 5
00000201:HAProxyLocalStats.srvhdr[0025:ffffffff]: Transfer-Encoding: chunked
00000201:HAProxyLocalStats.srvcls[0025:ffffffff]
tid:2 LOCKED srv: alias_srv4 process_chk_conn
tid:2 UNLOCK srv: alias_srv4 process_chk_conn
tid:2 LOCKED srv: alias_srv5 process_chk_conn
tid:2 UNLOCK srv: alias_srv5 process_chk_conn
tid:2 LOCKED srv: alias_srv6 process_chk_conn
tid:2 UNLOCK srv: alias_srv6 process_chk_conn
tid:2 LOCKED srv: alias_srv7 process_chk_conn
tid:2 UNLOCK srv: alias_srv7 process_chk_conn
tid:2 LOCKED srv: alias_srv2 process_chk_conn
tid:2 UNLOCK srv: alias_srv2 process_chk_conn
tid:2 LOCKED srv: alias_srv3 process_chk_conn
tid:2 UNLOCK srv: alias_srv3 process_chk_conn
[Switching to LWP 100730 of process 15238]
0x00000000004ab6bc in snr_check_ip_callback (srv=0x8022d0400, ip=0x80288e394,
ip_family=0x7fffffffe207 "\002\224\343\210\002\b")
at src/server.c:3781
3781 HA_SPIN_LOCK(SERVER_LOCK, &tmpsrv->lock);
(gdb) info threads
Id Target Id Frame
* 1 LWP 100730 of process 15238 0x00000000004ab6bc in snr_check_ip_callback
(srv=0x8022d0400, ip=0x80288e394,
ip_family=0x7fffffffe207 "\002\224\343\210\002\b") at src/server.c:3781
2 LWP 100924 of process 15238 0x00000000004ab6b4 in snr_check_ip_callback
(srv=0x8022cb000, ip=0x80288e394,
ip_family=0x7fffdfffda97 "\002\224\343\210\002\b") at src/server.c:3781
3 LWP 100925 of process 15238 0x000000000051ef21 in dns_resolve_recv
(dgram=0x8022483c0) at src/dns.c:1646
(gdb) bt full
#0 0x00000000004ab6bc in snr_check_ip_callback (srv=0x8022d0400,
ip=0x80288e394, ip_family=0x7fffffffe207 "\002\224\343\210\002\b")
at src/server.c:3781
__pl_r = 4294967300
tmpsrv = 0x8022cb000
be = 0x8022bb400
#1 0x000000000051d4ea in dns_get_ip_from_response (dns_p=0x802237a80,
dns_opts=0x8022d0be0, currentip=0x8022d079c, currentip_sin_family=2,
newip=0x7fffffffe2e8, newip_sin_family=0x7fffffffe2e4, owner=0x8022d0400)
at src/dns.c:1041
ip = 0x80288e394
ip_type = 2 '\002'
record = 0x80288e280
family_priority = 28
currentip_found = 0
newip4 = 0x0
newip6 = 0x0
currentip_sel = 0
j = 0
score = 0
max_score = -1
#2 0x00000000004ab33e in snr_resolution_cb (requester=0x8022a8e70,
nameserver=0x0) at src/server.c:3682
s = 0x8022d0400
resolution = 0x802237a00
serverip = 0x8022d079c
firstip = 0x0
server_sin_family = 2
firstip_sin_family = 0
ret = 8603200
chk = 0x80086d208
has_no_ip = 0
#3 0x000000000051ce11 in dns_trigger_resolution (req=0x8022a8e70) at
src/dns.c:353
resolvers = 0x802232600
res = 0x802237a00
exp = 2058800877
#4 0x00000000004fff5b in chk_report_conn_err (check=0x8022d0840, errno_bck=61,
expired=0) at src/checks.c:677
cs = 0x802893060
---Type <return> to continue, or q <return> to quit---
conn = 0x802694000
err_msg = 0x8022651c0 "Connection refused"
chk = 0x80086d218
step = 8
comment = 0x83be80 <fd_cache_local> "#"
#5 0x00000000004fe2a6 in event_srv_chk_w (cs=0x802893060) at src/checks.c:727
conn = 0x802694000
check = 0x8022d0840
s = 0x8022d0400
t = 0x80222aa00
#6 0x000000000059ff9f in mux_pt_send (conn=0x802694000) at src/mux_pt.c:109
cs = 0x802893060
#7 0x000000000058196b in conn_fd_handler (fd=35) at src/connection.c:130
conn = 0x802694000
flags = 0
#8 0x00000000005a25f0 in fdlist_process_cached_events (fdlist=0x83be80
<fd_cache_local>) at src/fd.c:434
fd = 35
old_fd = 35
e = 114
#9 0x00000000005a213c in fd_process_cached_events () at src/fd.c:451
No locals.
#10 0x0000000000512fdf in run_poll_loop () at src/haproxy.c:2428
next = 2058791770
exp = 2058791770
#11 0x00000000005103b0 in run_thread_poll_loop (data=0x80227b3b0) at
src/haproxy.c:2463
start_lock = 0
ptif = 0x82e898 <per_thread_init_list>
ptdf = 0x7fffffffead0
#12 0x000000000050cfb9 in main (argc=4, argv=0x7fffffffeaa8) at
src/haproxy.c:3053
tids = 0x80227b3b0
threads = 0x80235b280
i = 3
err = 0
retry = 200
limit = {rlim_cur = 6063, rlim_max = 6063}
errmsg =
"\000\352\377\377\377\177\000\000\320\352\377\377\377\177\000\000\250\352\377\377\377\177\000\000\004\000\000\000\000\000\000\000\323\063\002\234b\235\246\202P=\347\001\b\000\000\000\240\352\377\377\377\177\000\000\320\352\377\377\377\177\000\000\250\352\377\377\377\177\000\000\004\000\000\000\000\000\000\000@\352\377\377\377\177\000\000R\201\300\001\b\000\000\000\001\000\000"
pidfd = -1
(gdb) thread 2
[Switching to thread 2 (LWP 100924 of process 15238)]
#0 0x00000000004ab6b4 in snr_check_ip_callback (srv=0x8022cb000,
ip=0x80288e394, ip_family=0x7fffdfffda97 "\002\224\343\210\002\b") at
src/server.c:3781
3781 HA_SPIN_LOCK(SERVER_LOCK, &tmpsrv->lock);
(gdb) info threads
Id Target Id Frame
1 LWP 100730 of process 15238 0x00000000004ab6bc in snr_check_ip_callback
(srv=0x8022d0400, ip=0x80288e394,
ip_family=0x7fffffffe207 "\002\224\343\210\002\b") at src/server.c:3781
* 2 LWP 100924 of process 15238 0x00000000004ab6b4 in snr_check_ip_callback
(srv=0x8022cb000, ip=0x80288e394,
ip_family=0x7fffdfffda97 "\002\224\343\210\002\b") at src/server.c:3781
3 LWP 100925 of process 15238 0x000000000051ef21 in dns_resolve_recv
(dgram=0x8022483c0) at src/dns.c:1646
(gdb) bt full
#0 0x00000000004ab6b4 in snr_check_ip_callback (srv=0x8022cb000,
ip=0x80288e394, ip_family=0x7fffdfffda97 "\002\224\343\210\002\b") at
src/server.c:3781
__pl_l = 0x8022d0d98
__pl_r = 4294967300
tmpsrv = 0x8022d0400
be = 0x8022bb400
#1 0x000000000051d4ea in dns_get_ip_from_response (dns_p=0x802237a80,
dns_opts=0x8022cb7e0, currentip=0x8022cb39c, currentip_sin_family=2,
newip=0x7fffdfffdb78, newip_sin_family=0x7fffdfffdb74, owner=0x8022cb000)
at src/dns.c:1041
ip = 0x80288e394
ip_type = 2 '\002'
record = 0x80288e280
family_priority = 28
currentip_found = 0
newip4 = 0x0
newip6 = 0x0
currentip_sel = 0
j = 0
score = 0
max_score = -1
#2 0x00000000004ab33e in snr_resolution_cb (requester=0x8022a8d20,
nameserver=0x0) at src/server.c:3682
s = 0x8022cb000
resolution = 0x802237a00
serverip = 0x8022cb39c
firstip = 0x0
server_sin_family = 2
firstip_sin_family = 0
ret = 8603200
chk = 0x800870218
has_no_ip = 0
#3 0x000000000051ce11 in dns_trigger_resolution (req=0x8022a8d20) at
src/dns.c:353
resolvers = 0x802232600
res = 0x802237a00
exp = 2058800877
#4 0x00000000004fff5b in chk_report_conn_err (check=0x8022cb440, errno_bck=0,
expired=1) at src/checks.c:677
cs = 0x802691060
conn = 0x8028ab180
err_msg = 0x80261c7c0 ""
chk = 0x800870208
step = 8
comment = 0x100000000 <error: Cannot access memory at address
0x100000000>
#5 0x0000000000504d6c in process_chk_conn (t=0x80222a5a0) at src/checks.c:2226
check = 0x8022cb440
s = 0x8022cb000
cs = 0x802691060
conn = 0x8028ab180
rv = 0
ret = -536879776
expired = 1
#6 0x0000000000504087 in process_chk (t=0x80222a5a0) at src/checks.c:2291
check = 0x8022cb440
#7 0x00000000005958c1 in process_runnable_tasks () at src/task.c:320
t = 0x80222a5a0
i = 0
max_processed = 199
local_tasks = {0x80222a5a0, 0x800870350, 0x3e8, 0x7ab6ab0e7ab6ab2c,
0x7fffdfffdf10, 0x594e73 <wake_expired_tasks+355>, 0x100000002, 0x0, 0x0, 0x0,
0x83be10 <applet_active_lock>, 0x0, 0x0, 0x7fffdfffdee8,
0x7fffdfffdee8, 0xfffffffffffffffd}
local_tasks_count = 1
final_tasks_count = 0
#8 0x0000000000512e0b in run_poll_loop () at src/haproxy.c:2401
next = 2058791724
---Type <return> to continue, or q <return> to quit---
exp = 2058791724
#9 0x00000000005103b0 in run_thread_poll_loop (data=0x80227b3b4) at
src/haproxy.c:2463
start_lock = 0
ptif = 0x82e898 <per_thread_init_list>
ptdf = 0x800c6b7cc
#10 0x0000000800c66bc5 in ?? () from /lib/libthr.so.3
No symbol table info available.
#11 0x0000000000000000 in ?? ()
No symbol table info available.
Backtrace stopped: Cannot access memory at address 0x7fffdfffe000
(gdb) thread 3
[Switching to thread 3 (LWP 100925 of process 15238)]
#0 0x000000000051ef21 in dns_resolve_recv (dgram=0x8022483c0) at src/dns.c:1646
1646 HA_SPIN_LOCK(SERVER_LOCK, &s->lock);
(gdb) info threads
Id Target Id Frame
1 LWP 100730 of process 15238 0x00000000004ab6bc in snr_check_ip_callback
(srv=0x8022d0400, ip=0x80288e394,
ip_family=0x7fffffffe207 "\002\224\343\210\002\b") at src/server.c:3781
2 LWP 100924 of process 15238 0x00000000004ab6b4 in snr_check_ip_callback
(srv=0x8022cb000, ip=0x80288e394,
ip_family=0x7fffdfffda97 "\002\224\343\210\002\b") at src/server.c:3781
* 3 LWP 100925 of process 15238 0x000000000051ef21 in dns_resolve_recv
(dgram=0x8022483c0) at src/dns.c:1646
(gdb) bt full
#0 0x000000000051ef21 in dns_resolve_recv (dgram=0x8022483c0) at src/dns.c:1646
s = 0x8022cb000
ns = 0x802248000
tmpns = 0x802248000
resolvers = 0x802232600
res = 0x802237a00
query = 0x802237ab0
buf =
"\t\375\205\200\000\001\000\002\000\000\000\001\017dns_server_name\003pfs\005local\000\000\001\000\001\300\f\000\001\000\001\000\000\016\020\000\004\300\250\b\036\300\f\000\001\000\001\000\000\016\020\000\004\300\250\b\035\000\000)\020",
'\000' <repeats 3705 times>...
bufend = 0x7fffdfdfae16 ""
fd = 30
buflen = 86
dns_resp = 0
max_answer_records = 83
query_id = 2557
eb = 0x802237a3c
req = 0x8022a8d20
#1 0x00000000005bec3a in dgram_fd_handler (fd=30) at src/proto_udp.c:28
dgram = 0x8022483c0
#2 0x00000000005a25f0 in fdlist_process_cached_events (fdlist=0x83c360
<fd_cache>) at src/fd.c:434
fd = 30
old_fd = 30
e = 39
#3 0x00000000005a214b in fd_process_cached_events () at src/fd.c:452
No locals.
#4 0x0000000000512fdf in run_poll_loop () at src/haproxy.c:2428
next = 2058800895
exp = 2058800895
#5 0x00000000005103b0 in run_thread_poll_loop (data=0x80227b3b8) at
src/haproxy.c:2463
start_lock = 0
ptif = 0x82e898 <per_thread_init_list>
ptdf = 0x800c6b7cc
#6 0x0000000800c66bc5 in ?? () from /lib/libthr.so.3
No symbol table info available.
#7 0x0000000000000000 in ?? ()
No symbol table info available.
Backtrace stopped: Cannot access memory at address 0x7fffdfdfd000
