> On May 10, 2017, at 04:51, Emeric Brun <eb...@haproxy.com> wrote: > >> It looks like the main process stalls at DH_free(local_dh_1024) (part of >> __ssl_sock_deinit). Not sure why but I will debug and report back. >> >> Thanks, > > I experienced the same issue (stalled on a futex) if i run haproxy in > foreground and trying to kill it with kill -USR1. > > With this conf (dh param and ssl-async are disabled) > global > # tune.ssl.default-dh-param 2048 > ssl-engine qat > # ssl-async > nbproc 1
It looks like that the stall on futex issue is related to DH_free() calling ENGINE_finish in openssl 1.1: https://github.com/openssl/openssl/blob/master/crypto/dh/dh_lib.c#L109 (gdb) bt #0 __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132 #1 0x00007fa1582c5571 in pthread_rwlock_wrlock () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_rwlock_wrlock.S:118 #2 0x00007fa158a58559 in CRYPTO_THREAD_write_lock () from /tmp/openssl_1.1.0_install/lib/libcrypto.so.1.1 #3 0x00007fa1589d8800 in ENGINE_finish () from /tmp/openssl_1.1.0_install/lib/libcrypto.so.1.1 #4 0x00007fa158975e76 in DH_free () from /tmp/openssl_1.1.0_install/lib/libcrypto.so.1.1 #5 0x0000000000417c78 in free_dh () at src/ssl_sock.c:7905 #6 0x00007fa1591d58ce in _dl_fini () at dl-fini.c:254 #7 0x00007fa158512511 in __run_exit_handlers (status=0, listp=0x7fa15888e688, run_list_atexit=true) at exit.c:78 #8 0x00007fa158512595 in __GI_exit (status=<optimized out>) at exit.c:100 #9 0x0000000000408814 in main (argc=4, argv=0x7ffe72188548) at src/haproxy.c:2235 Openssl 1.1 has changed the way ENGINE_cleanup works: https://www.openssl.org/docs/man1.1.0/crypto/ENGINE_cleanup.html "From OpenSSL 1.1.0 it is no longer necessary to explicitly call ENGINE_cleanup and this function is deprecated. Cleanup automatically takes place at program exit." I suppose by the time the destructor __ssl_sock_deinit is called, engine-related cleanup are already done by openssl and ENGINE_finish (from DH_free) stalls on a non-existing write lock. I have a workaround which moves the DH_free logic out of the destructor __ssl_sock_deinit, and right before process exit. With the workaround I no longer see the stall issue. I am not sure whether it is optimal solution though. Let me know. Thanks, Grant
