[ANNOUNCE] haproxy-1.8-rc3
Hi, things are getting much better. We fixed a large number of remaining issues in the multi-threaded code (mostly unmatched locks), and various issues in the HTTP/2 code causing some streams either time out or some connections to be closed before the end of the response could be transmitted. There were also some issues in the HTTP/1 response parser used by the HTTP/2 gateway causing spinning loops on certain invalid responses such as status codes made of more than 3 digits, or on chunked responses filling the buffer. Also, the HTTP/1 parser now properly blocks "PRI" requests which are in fact caused by an HTTP/2 preface sent to a TCP frontend relaying to an HTTP backend. There was an issue with the multi-threaded task scheduler converging in O(N) when long series of tasks were running on the same thread, as triggered with HTTP/2 benchmarks. This was addressed so that it now does O(log(N)) again. So if you have run some benchmarks of H/2 with multithreaded and were surprised with some low performance results, you'll have to run them again :-) It's suspected that the applets scheduler will need the same change by the way, because while it used to endure little stress, with the cache that may change quite a bit. The code is now expected to build fine again on Solaris since SPIN_LOCK macros were renamed to HA_SPIN_LOCK, and the server-side 0-rtt TLS-1.3 code should now work. In master-worker mode, the pid file now only reports the parent's pid, which is more consistent with what is done by most other deamons and is more friendly to many tools. Nothing changed for the legacy multi-process mode however. I've run a number of tests on this one and could not freeze it nor crash it anymore. It even survived 100 million stats requests over H/2 with threads enabled without any error, something which previously would cause a few timeouts or spinning loops. I'm going to deploy 1.8-rc3 with threads enabled on haproxy.org now and watch it a little bit. We still have some cleanups to do in the code, and we have more or less decided what to do to address the HTTP2/cache/filters incompatibilities so hopefully cache+HTTP/2 will work fine together in rc4. While I used to say "be extremely careful" till rc2, I'd now say that you may want to give it a try on a single server if you're able to quickly take it out or roll back in case of outage. Please at least try to collect a core file if you see it crash, as there's no more known case where this is expected to happen. And please keep in mind that the HTTP/2 and multi-thread features are still experimental, so in case of trouble, just disable H2 and/or threads and see if the issue persists. Please find the usual URLs below : Site index : http://www.haproxy.org/ Discourse: http://discourse.haproxy.org/ Sources : http://www.haproxy.org/download/1.8/src/ Git repository : http://git.haproxy.org/git/haproxy-1.8.git/ Git Web browsing : http://git.haproxy.org/?p=haproxy-1.8.git Changelog: http://www.haproxy.org/download/1.8/src/CHANGELOG Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/ PS: I messed up during the first upload and force-pushed it again after checking in the logs that nobody tried to pull it. In case you have an automated mirror that reports an error, it's my fault and you'll have to fix it by hand. The code doesn't differ at all, it's just that the last commit with the changelog used to happen twice. Willy --- Complete changelog : Christopher Faulet (4): BUILD: threads: Rename SPIN/RWLOCK macros using HA_ prefix BUILD: enable USE_THREAD for Solaris build. BUG/MEDIUM: stream-int: Don't loss write's notifs when a stream is woken up BUG/MINOR: pattern: Rely on the sample type to copy it in pattern_exec_match Daniel Schneller (1): DOC: Add note about encrypted password CPU usage Emeric Brun (2): BUG/MEDIUM: splice/threads: pipe reuse list was not protected. BUG/MINOR: comp: fix compilation warning compiling without compression. Olivier Houchard (7): BUILD: use MAXPATHLEN instead of NAME_MAX. BUG/MINOR: dns: Don't try to get the server lock if it's already held. BUG/MINOR: dns: Don't lock the server lock in snr_check_ip_callback(). BUG/MINOR; ssl: Don't assume we have a ssl_bind_conf because a SNI is matched. MINOR: ssl: Handle session resumption with TLS 1.3 MINOR: ssl: Spell 0x10101000L correctly. MINOR: ssl: Handle sending early data to server. William Lallemand (4): MINOR: add master-worker in the warning about nbproc MINOR: mworker: allow pidfile in mworker + foreground MINOR: mworker: write parent pid in the pidfile MINOR: mworker: do not store child pid anymore in the pidfile Willy Tarreau (56): BUG/MAJOR: threads/checks: add 4 missing spin_unlock() in various functions BUG/MAJOR: threads/server: missing unlock in CLI fqdn parser BUG/MINOR: cli: do
HAProxy fails to compile against BoringSSL since 1.8-rc1
Hi there, I maintain a Docker-based build of HAProxy built against BoringSSL, tracking the BoringSSL version in Google Chrome: https://github.com/JayH5/docker-haproxy-boringssl I'm not really using this for anything... it's mostly just for fun and to try out TLS 1.3. This used to build okay on the 1.8 branch with the 1.8-dev releases, but since 1.8-rc1, the build has broken with errors as follows: gcc -Iinclude -Iebtree -Wall -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement -fwrapv -Wno-null-dereference -Wno-unused-label -DCONFIG_HAP_LINUX_SPLICE -DTPROXY -DCONFIG_HAP_LINUX_TPROXY -DCONFIG_HAP_CRYPT -DUSE_ZLIB -DENABLE_POLL -DENABLE_EPOLL -DUSE_CPU_AFFINITY -DASSUME_SPLICE_WORKS -DUSE_ACCEPT4 -DNETFILTER -DUSE_THREAD -DUSE_OPENSSL -I/usr/local/boringssl/include -DUSE_SYSCALL_FUTEX -DUSE_LUA -I/usr/local/lua/include -I/usr/local/lua/include -DUSE_PCRE2 -DPCRE2_CODE_UNIT_WIDTH=8 -I/usr/include -DUSE_PCRE2_JIT -DCONFIG_HAPROXY_VERSION=\"1.8-rc3-34650d5\" -DCONFIG_HAPROXY_DATE=\"2017/11/11\" -c -o src/hlua.o src/hlua.c src/ssl_sock.c: In function 'ctx_set_TLSv10_func': src/ssl_sock.c:1956:20: warning: implicit declaration of function 'SSL_CTX_set_ssl_version' [-Wimplicit-function-declaration] c == SET_SERVER ? SSL_CTX_set_ssl_version(ctx, TLSv1_server_method()) ^~~ src/ssl_sock.c: In function 'ssl_sock_switchctx_cbk': src/ssl_sock.c:2271:64: error: 'SET_MIN' undeclared (first use in this function) methodVersions[conf->ssl_methods.min].ssl_set_version(ssl, SET_MIN); ^~~ src/ssl_sock.c:2271:64: note: each undeclared identifier is reported only once for each function it appears in src/ssl_sock.c:2272:64: error: 'SET_MAX' undeclared (first use in this function) methodVersions[conf->ssl_methods.max].ssl_set_version(ssl, SET_MAX); ^~~ make: *** [Makefile:896: src/ssl_sock.o] Error 1 make: *** Waiting for unfinished jobs I haven't really had the time/energy to properly dig through things but I think there are some problems with the ifdefs in ssl_sock.c. I thought it might be worth reporting before the final 1.8 version is released. An example of the full logs of a Travis build are here: https://travis-ci.org/JayH5/docker-haproxy-boringssl/builds/300625108 Thank you for a very useful piece of software! Jamie
Re: [ANNOUNCE] haproxy-1.8-rc3
Great ;-)
Updated on docker hub.
https://hub.docker.com/r/me2digital/haproxy18/
###
HA-Proxy version 1.8-rc3-34650d5 2017/11/11
Copyright 2000-2017 Willy Tarreau
Build options :
TARGET = linux2628
CPU = generic
CC = gcc
CFLAGS = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement
-fwrapv -Wno-unused-label
OPTIONS = USE_LINUX_SPLICE=1 USE_GETADDRINFO=1 USE_ZLIB=1
USE_REGPARM=1 USE_OPENSSL=1 USE_LUA=1 USE_PCRE=1 USE_PCRE_JIT=1
USE_TFO=1
Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents =
200
Built with OpenSSL version : OpenSSL 1.0.2k-fips 26 Jan 2017
Running on OpenSSL version : OpenSSL 1.0.2k-fips 26 Jan 2017
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : SSLv3 TLSv1.0 TLSv1.1 TLSv1.2
Built with Lua version : Lua 5.3.4
Built with transparent proxy support using: IP_TRANSPARENT
IPV6_TRANSPARENT IP_FREEBIND
Built with network namespace support.
Built with zlib version : 1.2.7
Running on zlib version : 1.2.7
Compression algorithms supported : identity("identity"),
deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Encrypted password support via crypt(3): yes
Built with PCRE version : 8.32 2012-11-30
Running on PCRE version : 8.32 2012-11-30
PCRE library supports JIT : yes
Built with multi-threading support.
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
Available filters :
[SPOE] spoe
[COMP] compression
[TRACE] trace
###
-- Originalnachricht --
Von: "Willy Tarreau"
An: haproxy@formilux.org
Gesendet: 11.11.2017 09:34:13
Betreff: [ANNOUNCE] haproxy-1.8-rc3
Hi,
things are getting much better. We fixed a large number of remaining
issues in the multi-threaded code (mostly unmatched locks), and various
issues in the HTTP/2 code causing some streams either time out or some
connections to be closed before the end of the response could be
transmitted. There were also some issues in the HTTP/1 response parser
used by the HTTP/2 gateway causing spinning loops on certain invalid
responses such as status codes made of more than 3 digits, or on
chunked
responses filling the buffer. Also, the HTTP/1 parser now properly
blocks
"PRI" requests which are in fact caused by an HTTP/2 preface sent to a
TCP frontend relaying to an HTTP backend.
There was an issue with the multi-threaded task scheduler converging
in O(N) when long series of tasks were running on the same thread, as
triggered with HTTP/2 benchmarks. This was addressed so that it now
does O(log(N)) again. So if you have run some benchmarks of H/2 with
multithreaded and were surprised with some low performance results,
you'll have to run them again :-) It's suspected that the applets
scheduler will need the same change by the way, because while it used
to endure little stress, with the cache that may change quite a bit.
The code is now expected to build fine again on Solaris since SPIN_LOCK
macros were renamed to HA_SPIN_LOCK, and the server-side 0-rtt TLS-1.3
code should now work.
In master-worker mode, the pid file now only reports the parent's pid,
which is more consistent with what is done by most other deamons and is
more friendly to many tools. Nothing changed for the legacy
multi-process
mode however.
I've run a number of tests on this one and could not freeze it nor
crash it anymore. It even survived 100 million stats requests over H/2
with threads enabled without any error, something which previously
would cause a few timeouts or spinning loops.
I'm going to deploy 1.8-rc3 with threads enabled on haproxy.org now and
watch it a little bit.
We still have some cleanups to do in the code, and we have more or less
decided what to do to address the HTTP2/cache/filters incompatibilities
so hopefully cache+HTTP/2 will work fine together in rc4.
While I used to say "be extremely careful" till rc2, I'd now say that
you may want to give it a try on a single server if you're able to
quickly take it out or roll back in case of outage. Please at least try
to collect a core file if you see it crash, as there's no more known
case where this is expected to happen. And please keep in mind that the
HTTP/2 and multi-thread features are still experimental, so in case of
trouble, just disable H2 and/or threads and see if the issue persists.
Please find the usual URLs below :
Site index : http://www.haproxy.org/
Discourse: http://discourse.haproxy.org/
Sources : http://www.haproxy.org/download/1.8/src/
Git repository : http://git.haproxy.org/git/haproxy-1.8.git/
Git Web browsing : http://git.haproxy.org/?p=haproxy-1.8.git
Changelog: http://www.haproxy.org/download/1.8/src/CHANGELOG
Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/
PS: I messed up during the first upload and force-pushed it again after
checking in the logs that
