Hi,
HAProxy 1.9.7 was released on 2019/04/25. It added 100 new commits after version
1.9.6.
After a month since the last release, a huge number of bugs were addressed into
this release. The most significant are fixes of 100% CPU usage issues. Olivier
and Willy worked hard on it. They fixed several bugs in the task scheduler and
improved it. Another issue concerned the streams and the way aborts was
handled. The flag reporting the abort on the stream-interface was cleared too
early. So the stream was not properly released, looping infinitely on
itself. Thanks to Maciej for helping Olivier investigating on this
issue. Finally, two bugs leading to a loop were fixed in the H2 multiplexer. The
first one occurred when an incomplete frame was received and no more data was
available. The second one when the subscription to the send events was made
twice, which should never happen. This last issue was also able to lead to
random crashes.
Willy, with the help of Maksim, fixed a locking problem in the round-robin LB
causing occasional crashes of HAProxy because of a divide by zero when the next
server's position was computed. Willy also fixed an AB/BA locking issue in the
listener's accept code thanks to Olivier and his self-locked list. At the end,
all this part was reworked. The lock on the listener queue disappeared and the
accept code was significantly improved. As mentioned, Olivier implemented
self-locked lists. It is now safe to manipulate such lists with multiple threads
without having to worry about concurrency issues.
A very old bug on how HTTP sample fetches work was fixed. All HTTP sample
fetches were buggy because the channel used was chosen depending on the sample
direction and not on the keyword really used. The request channel was used when
called during the request analysis and the response one was used when called
during the response analysis, regardless the sample really called. It could
cause a whole bunch of bugs, from undefined behavior because the data were
extracted from the wrong buffer to crash of HAProxy.
Many bugs in the HTX part were fixed. The most significant was a problem of data
corruption when an header was added while some data were already received. It
was addressed by defragmenting the HTX message. It is a quick and dirty fix,
waiting for a better solution. The TCP splicing was fixed in the H1
multiplexer. The handling of the CONNECT method was refactored to work when h1
and h2 are mixed (an h1 client sending requests to an h2 server or this
opposite). And finally, the option "http-tunnel" was deprecated in HTX. It is
still supported in legacy HTTP, but in HTX, this options is now ignored.
Otherwise, as always, there was a bunch of bugfixes and changes here and
there. The most noticeable are:
- A memory leak was fixed in the map converter. And now when a map file is
loaded, the default value is parsed only when it is present. This fixes
segfaults at parsing time when no default value is provided.
- Pattern IDs are now assigned after checking the config validity. It fixes a
bug where some map identifiers were not assigned (appearing as -1 in show
map). Thanks to Pavlos to report this bug.
- Several bugs was fixed on the SPOE, mainly on the way fragmented frames was
handled internally.
- A bug was fixed in the peers. Peer sessions were not always cleanly reset
on release, resulting in a bad state for new sessions. In addition, the CLI
command "show peers" was added. It dumps info about the peers configured in
"peers" sections.
- Nenad fixed a security issue in the TLS ticket key rotation. Whenever
HAProxy was reloaded with rotated keys, the resumption would be broken for
previous encryption key. CVE-2019-11323 was assigned to this bug.
- Build of HAProxy on AIX 5.1 was fixed.
- Some small improvements were brought to the master-worker mode. The busy
polling was disabled on the master process to not waste a dedicated
processor on it. And an unambiguous status is now set on exit.
- The global directive "set-dumpable" was added to enable core dumps. It's
always a pain to get a core dump when enabling user/group setting (which
disables the dumpable flag on Linux), when using a chroot and/or when
HAProxy is started by a service management tool which requires complex
operations to just raise the core dump limit. It becomes much easier to
retrieve a core file by using this new directive. Temporarily moving the
chroot to a user-writable place generally enough.
- Missing locks was added in set-map and add-acl HTTP rules.
- The multithreading mode is now enabled on wurlf. And it is now compatible
with the new buffer API, so it builds again on 1.9.
WARNING: Device detection modules (51Degrees, DeviceAtlas and Wurfl) are not
compatible with the HTX mode. Respective maintainers works on it. But for this
release, don't enable the HTX if you need to use one of them.
There is still an unresolved issue when the SSL is heavily used with multiple
threads. It is unclear for now what the problem is, but an extra latency has
been observed by a user. After digging, we suspect a problem with OpenSSL. You
can have more details on this issue on Github (issue #67).
Please find the usual URLs below :
Site index : http://www.haproxy.org/
Discourse : http://discourse.haproxy.org/
Slack channel : https://slack.haproxy.org/
Issue tracker : https://github.com/haproxy/haproxy/issues
Sources : http://www.haproxy.org/download/1.9/src/
Git repository : http://git.haproxy.org/git/haproxy-1.9.git/
Git Web browsing : http://git.haproxy.org/?p=haproxy-1.9.git
Changelog : http://www.haproxy.org/download/1.9/src/CHANGELOG
Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/
---
Complete changelog :
Christopher Faulet (28):
BUG/MINOR: htx: Preserve empty HTX messages with an unprocessed parsing
error
BUG/MEDIUM: spoe: Queue message only if no SPOE applet is attached to
the stream
BUG/MEDIUM: spoe: Return an error if nothing is encoded for fragmented
messages
BUG/MINOR: spoe: Be sure to set tv_request when each message fragment is
encoded
BUG/MEDIUM: htx: Defrag if blocks position is changed and the payloads
wrap
BUG/MEDIUM: htx: Don't crush blocks payload when append is done on a
data block
MEDIUM: htx: Deprecate the option 'http-tunnel' and ignore it in HTX
MINOR: proto_htx: Don't adjust transaction mode anymore in HTX analyzers
BUG/MEDIUM: htx: Fix the process of HTTP CONNECT with h2 connections
MINOR: mux-h1: Simplify handling of 1xx responses
BUG/MINOR: mux-h1: Handle the flag CS_FL_KILL_CONN during a shutdown
read/write
BUG/MINOR: mux-h1: Process input even if the input buffer is empty
BUG/MINOR: mux-h1: Don't switch the parser in busy mode if other side
has done
BUG/MEDIUM: mux-h1: Notify the stream waiting for TCP splicing if ibuf
is empty
BUG/MEDIUM: mux-h1: Enable TCP splicing to exchange data only
MINOR: mux-h1: Handle read0 during TCP splicing
BUG/MEDIUM: htx: Don't return the start-line if the HTX message is empty
BUG/MAJOR: http_fetch: Get the channel depending on the keyword used
BUG/MINOR: http_fetch/htx: Allow permissive sample prefetch for the HTX
BUG/MEDIUM: h1: Don't parse chunks CRLF if not enough data are available
BUG/MEDIUM: thread/http: Add missing locks in set-map and add-acl HTTP
rules
BUG/MEDIUM: stream: Don't request a server connection if a shutw was
scheduled
BUG/MINOR: 51d: Get the request channel to call
CHECK_HTTP_MESSAGE_FIRST()
BUG/MINOR: da: Get the request channel to call CHECK_HTTP_MESSAGE_FIRST()
BUG/MEDIUM: stream: Fix the way early aborts on the client side are
handled
BUG/MINOR: spoe: Don't systematically wakeup SPOE stream in the applet
handler
BUG/MAJOR: muxes: Use the HTX mode to find the best mux for HTTP proxies
only
BUG/MINOR: htx: Exclude TCP proxies when the HTX mode is handled during
startup
David Carlier (1):
BUILD/MINOR: listener: Silent a few signedness warnings.
Emeric Brun (3):
BUG/MEDIUM: peers: fix a case where peer session is not cleanly reset on
release.
MINOR: peers: adds counters on show peers about tasks calls.
MINOR: ssl/cli: async fd io-handlers printable on show fd
Frédéric Lécaille (2):
MINOR: peers: Add a new command to the CLI for peers.
DOC: update for "show peers" CLI command.
Ilya Shipitsin (1):
REGTESTS: exclude tests that require ssl, pcre if no such feature is
enabled
Nenad Merdanovic (2):
BUG/MEDIUM: map: Fix memory leak in the map converter
BUG/MINOR: ssl: Fix 48 byte TLS ticket key rotation
Olivier Houchard (15):
BUG/MEDIUM: h2: Don't attempt to recv from h2_process_demux if we
subscribed.
BUG/MEDIUM: streams: Don't remove the SI_FL_ERR flag in si_update_both().
BUG/MEDIUM: streams: Store prev_state before calling si_update_both().
BUG/MEDIUM: stream: Don't clear the stream_interface flags in
si_update_both.
BUG/MEDIUM: streams: Only re-run process_stream if we're in a connected
state.
BUG/MEDIUM: stream_interface: Don't bother doing chk_rcv/snd if not
connected.
MINOR: lists: Implement locked variations.
BUG/MEDIUM: lists: Properly handle the case we're removing the first elt.
MINOR: initcall: Don't forget to define the __start/stop_init_##stg
symbols.
BUG/MEDIUM: h2: Make sure we're not already in the send_list in
h2_subscribe().
BUG/MEDIUM: tasks: Make sure we modify global_tasks_mask with the
rq_lock.
MINOR: tasks: Don't consider we can wake task with tasklet_wakeup().
MEDIUM: tasks: No longer use rq.node.leaf_p as a lock.
MINOR: tasks: Don't set the TASK_RUNNING flag when adding in the tasklet
list.
BUG/MEDIUM: applets: Don't use task_in_rq().
Robin H. Johnson (1):
MINOR: skip get_gmtime where tm is unused
William Lallemand (4):
BUG/MINOR: cli: correctly handle abns in 'show cli sockets'
MINOR: cli: start addresses by a prefix in 'show cli sockets'
BUG/MINOR: mworker: don't exit with an ambiguous value
BUG/MINOR: mworker: ensure that we still quits with SIGINT
Willy Tarreau (40):
BUILD: makefile: work around an old bug in GNU make-3.80
BUILD: http: properly mark some struct as extern
BUILD: chunk: properly declare pool_head_trash as extern
BUILD: cache: avoid a build warning with some compilers/linkers
MINOR: tools: make memvprintf() never pass a NULL target to vsnprintf()
BUILD: re-implement an initcall variant without using executable sections
BUILD: makefile: fix build of IPv6 header on aix51
BUILD: makefile: add _LINUX_SOURCE_COMPAT to build on AIX-51
BUILD: Makefile: disable shared cache on AIX 5.1
BUILD: fix backport of initcall variant
BUILD: use inttypes.h instead of stdint.h
BUILD: connection: fix naming of ip_v field
BUG/MEDIUM: htx: fix random premature abort of data transfers
BUG/MEDIUM: pattern: assign pattern IDs after checking the config
validity
BUG/MEDIUM: task/threads: address a fairness issue between local and
global tasks
BUG/MINOR: tasks: make sure the first task to be queued keeps its nice
value
BUG/MINOR: threads: fix the process range of thread masks
BUG/MEDIUM: list: fix the rollback on addq in the locked liss
BUG/MEDIUM: list: fix LIST_POP_LOCKED's removal of the last pointer
BUG/MEDIUM: list: add missing store barriers when updating elements and
head
MINOR: list: make the delete and pop operations idempotent
BUG/MEDIUM: list: correct fix for LIST_POP_LOCKED's removal of last
element
BUG/MEDIUM: list: fix again LIST_ADDQ_LOCKED
BUG/MEDIUM: list: fix incorrect pointer unlocking in LIST_DEL_LOCKED()
MAJOR: listener: do not hold the listener lock in listener_accept()
BUG/MEDIUM: listener: use a self-locked list for the dequeue lists
BUG/MEDIUM: listener: make sure the listener never accepts too many conns
BUILD: htx: fix a used uninitialized warning on is_cookie2
BUG/MAJOR: lb/threads: fix insufficient locking on round-robin LB
MEDIUM: tasks: improve fairness between the local and global queues
BUILD: task/thread: fix single-threaded build of task.c
MEDIUM: tasks: only base the nice offset on the run queue depth
MINOR: tasks: restore the lower latency scheduling when niced tasks are
present
BUG/MEDIUM: tasks: Make sure we set TASK_QUEUED before adding a task to
the rq.
BUG/MAJOR: task: make sure never to delete a queued task
BUG/MINOR: mworker: disable busy polling in the master process
BUG/MEDIUM: maps: only try to parse the default value when it's present
BUG/MINOR: acl: properly detect pattern type SMP_T_ADDR
MINOR: init: add a "set-dumpable" global directive to enable core dumps
BUG/MAJOR: lb/threads: fix AB/BA locking issue in round-robin LB
paulborile (3):
BUILD: wurfl: build fix for 1.9/2.0 code base
MINOR: wurfl: enabled multithreading mode
DOC: wurfl: added point of contact in MAINTAINERS file
--
Christopher Faulet
