Hi Tim,
On Thu, Nov 09, 2017 at 01:00:22AM +0100, Tim Düsterhus wrote:
> Hi
>
> I get the following crash when running:
>
> [timwolla@/t/h/haproxy-1.8-rc2]./haproxy -V
> HA-Proxy version 1.8-rc2-a8d8d6e 2017/11/03
> Copyright 2000-2017 Willy Tarreau <wi...@haproxy.org>
>
> with the configuration at the bottom of this email as follows:
>
> root@node42:/tmp/haproxy/haproxy-1.8-rc2# ./haproxy -W -f
> /tmp/haproxy/haproxy-1.8-rc2/haproxy.cfg
> [WARNING] 312/004845 (31835) : Can't open server state file
> '/etc/haproxy/state/global': No such file or directory
> [WARNING] 312/004845 (31835) : Can't open server state file
> '/etc/haproxy/state/global': No such file or directory
>
> and then killing the master process using a SIGHUP:
>
> > *** Error in `./haproxy': munmap_chunk(): invalid pointer:
> > 0x0000000000515028 ***
> > ======= Backtrace: =========
> > /lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7fda72bbc7e5]
> > /lib/x86_64-linux-gnu/libc.so.6(cfree+0x1a8)[0x7fda72bc9698]
> > ./haproxy[0x4a28d0]
> > ./haproxy[0x4a2c4e]
> > ./haproxy[0x4a2f57]
> > ./haproxy[0x40b16f]
> > /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7fda72b65830]
> > ./haproxy[0x40c149]
> > ======= Memory map: ========
> > 00400000-00532000 r-xp 00000000 08:03 2765518
> > /tmp/haproxy/haproxy-1.8-rc2/haproxy
> > 00731000-00732000 r--p 00131000 08:03 2765518
> > /tmp/haproxy/haproxy-1.8-rc2/haproxy
> > 00732000-00749000 rw-p 00132000 08:03 2765518
> > /tmp/haproxy/haproxy-1.8-rc2/haproxy
> > 00749000-0074e000 rw-p 00000000 00:00 0
> > 01d7f000-01de3000 rw-p 00000000 00:00 0
> > [heap]
> > 7fda7272b000-7fda72741000 r-xp 00000000 08:03 528903
> > /lib/x86_64-linux-gnu/libgcc_s.so.1
> > 7fda72741000-7fda72940000 ---p 00016000 08:03 528903
> > /lib/x86_64-linux-gnu/libgcc_s.so.1
> > 7fda72940000-7fda72941000 rw-p 00015000 08:03 528903
> > /lib/x86_64-linux-gnu/libgcc_s.so.1
> > 7fda72941000-7fda72944000 r-xp 00000000 08:03 533578
> > /lib/x86_64-linux-gnu/libdl-2.23.so
> > 7fda72944000-7fda72b43000 ---p 00003000 08:03 533578
> > /lib/x86_64-linux-gnu/libdl-2.23.so
> > 7fda72b43000-7fda72b44000 r--p 00002000 08:03 533578
> > /lib/x86_64-linux-gnu/libdl-2.23.so
> > 7fda72b44000-7fda72b45000 rw-p 00003000 08:03 533578
> > /lib/x86_64-linux-gnu/libdl-2.23.so
> > 7fda72b45000-7fda72d05000 r-xp 00000000 08:03 533590
> > /lib/x86_64-linux-gnu/libc-2.23.so
> > 7fda72d05000-7fda72f05000 ---p 001c0000 08:03 533590
> > /lib/x86_64-linux-gnu/libc-2.23.so
> > 7fda72f05000-7fda72f09000 r--p 001c0000 08:03 533590
> > /lib/x86_64-linux-gnu/libc-2.23.so
> > 7fda72f09000-7fda72f0b000 rw-p 001c4000 08:03 533590
> > /lib/x86_64-linux-gnu/libc-2.23.so
> > 7fda72f0b000-7fda72f0f000 rw-p 00000000 00:00 0
> > 7fda72f0f000-7fda72f7d000 r-xp 00000000 08:03 541796
> > /lib/x86_64-linux-gnu/libpcre.so.3.13.2
> > 7fda72f7d000-7fda7317d000 ---p 0006e000 08:03 541796
> > /lib/x86_64-linux-gnu/libpcre.so.3.13.2
> > 7fda7317d000-7fda7317e000 r--p 0006e000 08:03 541796
> > /lib/x86_64-linux-gnu/libpcre.so.3.13.2
> > 7fda7317e000-7fda7317f000 rw-p 0006f000 08:03 541796
> > /lib/x86_64-linux-gnu/libpcre.so.3.13.2
> > 7fda7317f000-7fda73399000 r-xp 00000000 08:03 529753
> > /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
> > 7fda73399000-7fda73598000 ---p 0021a000 08:03 529753
> > /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
> > 7fda73598000-7fda735b4000 r--p 00219000 08:03 529753
> > /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
> > 7fda735b4000-7fda735c0000 rw-p 00235000 08:03 529753
> > /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
> > 7fda735c0000-7fda735c3000 rw-p 00000000 00:00 0
> > 7fda735c3000-7fda73621000 r-xp 00000000 08:03 524697
> > /lib/x86_64-linux-gnu/libssl.so.1.0.0
> > 7fda73621000-7fda73821000 ---p 0005e000 08:03 524697
> > /lib/x86_64-linux-gnu/libssl.so.1.0.0
> > 7fda73821000-7fda73825000 r--p 0005e000 08:03 524697
> > /lib/x86_64-linux-gnu/libssl.so.1.0.0
> > 7fda73825000-7fda7382c000 rw-p 00062000 08:03 524697
> > /lib/x86_64-linux-gnu/libssl.so.1.0.0
> > 7fda7382c000-7fda73844000 r-xp 00000000 08:03 530442
> > /lib/x86_64-linux-gnu/libpthread-2.23.so
> > 7fda73844000-7fda73a43000 ---p 00018000 08:03 530442
> > /lib/x86_64-linux-gnu/libpthread-2.23.so
> > 7fda73a43000-7fda73a44000 r--p 00017000 08:03 530442
> > /lib/x86_64-linux-gnu/libpthread-2.23.so
> > 7fda73a44000-7fda73a45000 rw-p 00018000 08:03 530442
> > /lib/x86_64-linux-gnu/libpthread-2.23.so
> > 7fda73a45000-7fda73a49000 rw-p 00000000 00:00 0
> > 7fda73a49000-7fda73a62000 r-xp 00000000 08:03 524466
> > /lib/x86_64-linux-gnu/libz.so.1.2.8
> > 7fda73a62000-7fda73c61000 ---p 00019000 08:03 524466
> > /lib/x86_64-linux-gnu/libz.so.1.2.8
> > 7fda73c61000-7fda73c62000 r--p 00018000 08:03 524466
> > /lib/x86_64-linux-gnu/libz.so.1.2.8
> > 7fda73c62000-7fda73c63000 rw-p 00019000 08:03 524466
> > /lib/x86_64-linux-gnu/libz.so.1.2.8
> > 7fda73c63000-7fda73c6c000 r-xp 00000000 08:03 526340
> > /lib/x86_64-linux-gnu/libcrypt-2.23.so
> > 7fda73c6c000-7fda73e6b000 ---p 00009000 08:03 526340
> > /lib/x86_64-linux-gnu/libcrypt-2.23.so
> > 7fda73e6b000-7fda73e6c000 r--p 00008000 08:03 526340
> > /lib/x86_64-linux-gnu/libcrypt-2.23.so
> > 7fda73e6c000-7fda73e6d000 rw-p 00009000 08:03 526340
> > /lib/x86_64-linux-gnu/libcrypt-2.23.so
> > 7fda73e6d000-7fda73e9b000 rw-p 00000000 00:00 0
> > 7fda73e9b000-7fda73ec1000 r-xp 00000000 08:03 524932
> > /lib/x86_64-linux-gnu/ld-2.23.so
> > 7fda7408d000-7fda74093000 rw-p 00000000 00:00 0
> > 7fda74095000-7fda740c0000 rw-p 00000000 00:00 0
> > 7fda740c0000-7fda740c1000 r--p 00025000 08:03 524932
> > /lib/x86_64-linux-gnu/ld-2.23.so
> > 7fda740c1000-7fda740c2000 rw-p 00026000 08:03 524932
> > /lib/x86_64-linux-gnu/ld-2.23.so
> > 7fda740c2000-7fda740c3000 rw-p 00000000 00:00 0
> > 7ffc98439000-7ffc9845a000 rw-p 00000000 00:00 0
> > [stack]
> > 7ffc98548000-7ffc9854a000 r--p 00000000 00:00 0
> > [vvar]
> > 7ffc9854a000-7ffc9854c000 r-xp 00000000 00:00 0
> > [vdso]
> > ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0
> > [vsyscall]
> > Aborted (core dumped)
> I will be providing a link to a .tar.gz archive with the compiled
> binary, core file and haproxy.cfg to Willy in private, because they are
> way to big to attach to a mailing list.
I'm pretty certain it's this bug that I introduced when adding the
indication of built-in thread support in the output of -vv, that I
fixed with this patch, because I had exactly the same report :
commit 6dbd3e963bc4f7c3fae545b623989afe5d68c198
Author: Willy Tarreau <w...@1wt.eu>
Date: Sun Nov 5 11:50:18 2017 +0100
BUG/MEDIUM: threads: don't try to free build option message on exit
Commit 0493149 ("MINOR: thread: report multi-thread support in haproxy -vv")
added information about thread support in haproxy -vv output but accidently
marked the message as "must_free" while it's a constant. This causes a segv
on the old process on clean exit if threads are enabled. It doesn't affect
the stability during operations however.
It is harmless (despite annoying for your tests) since it dies when freeing
everything on exit. If you want to get rid of it, it's fixed in the latest
daily snapshot.
Thanks!
Willy
