Hi,
HAProxy 2.9.10 was released on 2024/09/03. It added 52 new commits
after version 2.9.9.
This release groups all backportable fixes shipped in the 3.0.3 and
3.0.4. The main topic is about fixes of two issues affecting how the H2 mux
deals with incomplete frames:
- in one case, certain errors happening while processing an incomplete
frame did not lead to the termination of the connection, and would cause
endless wakeups to try to handle the error, preventing the process from
sleeping, thus eating CPU.
- another case, much harder to reproduce but also observed as actively
exploited in one case, can cause an endless loop in the h2_send()
function if a processing error requiring a GOAWAY is reported with an
almost full output buffer when no more progress can be made on the input
buffer due to an incomplete frame while many streams are transmitting
data in parallel in zero-copy mode. What happens in this case is that
the output buffer is cleared (due to the error) while still leaving the
full indication that prevents output data from being considered, and no
condition to exit the loop is met. In this case the loop will be
interrupted by the watchdog which will kill the process after two
seconds. A work-around consists in simply disabling zero-copy forwarding
for HTTP/2: "tune.h2.zero-copy-fwd-send off". This issue was assigned
CVE-2024-45506.
Other than that, the following issues were fixed:
- 16 threads (due to an area shared for two different things it would
start to dump threads from the 17th).
- JWT: the SSL library functions used to validate a token would leave
an error in the SSL stack, that will later be mistaken for an error
on another connection and cause it to be closed.
- a time-of-check/time-of-use (TOCTOU) issue in the queue processing
makes it rare but possible to leave a server with no connection yet
not take any traffic. It's more likely to happen with maxconn 1,
very hard at 2 and almost impossible at 3 or above.
- QUIC: there was a case which could produce crashes when built with the
aws-lc TLS library.
- OCSP: a memory allocation error while loading OCSP parameters could
leave the tree locked and freeze subsequent operations.
- some uploads to H2 servers could freeze due to the zero-copy forwarding
not always setting the END_STREAM flag on the last DATA frame (GH
#2665).
- it was possible to crash the process when performing an implicit
protocol upgrade (TCP to HTTP due to a transition from a TCP front to an
HTTP back) if an error happened on the connection just before the
transition.
- a crash could happen in mux-pt if an error happened on the connection
just before an abort that is going to emit a shutdown, and with a
pending wakeup that completes some work on a connection having no
transport layer anymore. This only affects TCP (e.g. peers and master
CLI; GH #2656).
- mux-h1 could repeat a 408 error multiple times in logs when failing to
send an empty message on a full output buffer. In this case, it would
attempt to close again every client timeout and produce a log each time
despite no data leaving.
- a bug in the bwlim filter that can make it forget to update its timeout
and loop fast when woken up with nothing to forward.
- the lack of filtering of empty transfer-encoding headers in the H1 mux
that was reported by Ben Kallus.
- the insufficient syntax checks on :method and :scheme H3 headers that
was reported by Yuki Mogi.
- a hang on lua CLI applets when trying to deal with their own buffers.
- a possible crash in deinit() (when quitting/reloading) when using e-mail
alerts after a fix that went into 2.9.9.
- a rare but possible deadlock (and crash) in QUIC after a recent fix for
a race condition in the CID tree, that also went into 2.9.9.
- an unreproduced race condition affecting the QUIC CID tree. The impact
might be a possible crash but it was only found in code review and never
reported nor reproduced.
- an issue in SPOE that can cause a thread to refrain from creating an
applet to connect outside, causing failures on requests processed on
this thread.
- a race in stick-tables starting from 2.9 where an element may be
accessed immediately after decrementing the refcount, sometimes allowing
it to be purged in parallel and causing crashes (GH #2611).
Finally a change that is not exactly an issue but addressing a problem
encountered in some setups:
- the hard limit on the number of file descriptors now defaults to about
1 million, in order to match what has been done for a very long time
on many distros, and that recently changed to 1 billion on some of
them, causing a huge startup time (or even a watchdog at boot) and a
massive memory usage.
And the rest is pretty minor. Everyone running on the 2.9.9 is encouraged to
update to 2.9.10. Many thanks to everyone helping us to improve haproxy !
Please find the usual URLs below :
Site index : https://www.haproxy.org/
Documentation : https://docs.haproxy.org/
Wiki : https://github.com/haproxy/wiki/wiki
Discourse : https://discourse.haproxy.org/
Slack channel : https://slack.haproxy.org/
Issue tracker : https://github.com/haproxy/haproxy/issues
Sources : https://www.haproxy.org/download/2.9/src/
Git repository : https://git.haproxy.org/git/haproxy-2.9.git/
Git Web browsing : https://git.haproxy.org/?p=haproxy-2.9.git
Changelog : https://www.haproxy.org/download/2.9/src/CHANGELOG
Dataplane API :
https://github.com/haproxytech/dataplaneapi/releases/latest
Pending bugs : https://www.haproxy.org/l/pending-bugs
Reviewed bugs : https://www.haproxy.org/l/reviewed-bugs
Code reports : https://www.haproxy.org/l/code-reports
Latest builds : https://www.haproxy.org/l/dev-packages
---
Complete changelog :
Amaury Denoyelle (11):
SCRIPTS: git-show-backports: do not truncate git-show output
BUG/MINOR: h3: fix crash on STOP_SENDING receive after GOAWAY emission
BUG/MINOR: mux-quic: fix crash on qcs SD alloc failure
BUG/MINOR: quic: fix BUG_ON() on Tx pkt alloc failure
BUG/MEDIUM: h3: ensure the ":method" pseudo header is totally valid
BUG/MEDIUM: h3: ensure the ":scheme" pseudo header is totally valid
BUG/MEDIUM: quic: fix race-condition in quic_get_cid_tid()
BUG/MINOR: quic: fix race condition in qc_check_dcid()
BUG/MINOR: quic: fix race-condition on trace for CID retrieval
BUG/MINOR: stick-table: fix crash for src_inc_gpc() without stkcounter
DOC: quic: fix default minimal value for max window size
Aurelien DARRAGON (2):
DOC: api/event_hdl: small updates, fix an example and add some precisions
BUG/MINOR: hlua: report proper context upon error in
hlua_cli_io_handler_fct()
Christopher Faulet (16):
BUG/MEDIUM: proxy: fix email-alert invalid free
BUG/MEDIUM: stick-table: Decrement the ref count inside lock to kill a
session
BUG/MINOR: h1: Fail to parse empty transfer coding names
BUG/MINOR: h1: Reject empty coding name as last transfer-encoding value
BUG/MEDIUM: h1: Reject empty Transfer-encoding header
BUG/MEDIUM: spoe: Be sure to create a SPOE applet if none on the current
thread
BUG/MEDIUM: bwlim: Be sure to never set the analyze expiration date in
past
BUG/MINOR: session: Eval L4/L5 rules defined in the default section
BUG/MINOR: server: Don't warn fallback IP is used during init-addr
resolution
BUG/MINOR: cli: Atomically inc the global request counter between CLI
commands
BUG/MEDIUM: jwt: Clear SSL error queue on error when checking the
signature
BUG/MEDIUM: mux-h2: Set ES flag when necessary on 0-copy data forwarding
BUG/MEDIUM: stream: Prevent mux upgrades if client connection is no
longer ready
BUG/MEDIUM: mux-pt: Never fully close the connection on shutdown
BUG/MEDIUM: cli: Always release back endpoint between two commands on the
mcli
BUG/MEDIUM: mux-h1: Properly handle empty message when an error is
triggered
Frederic Lecaille (4):
BUG/MINOR: quic: Lack of precision when computing K (cubic only cc)
BUG/MINOR: quic: unexploited retransmission cases for Initial pktns.
BUG/MINOR: quic: Too shord datagram during O-RTT handshakes (aws-lc only)
BUG/MINOR: Crash on O-RTT RX packet after dropping Initial pktns
Valentine Krasnobaeva (4):
DOC: configuration: update maxconn description
BUG/MEDIUM: ssl_sock: fix deadlock in ssl_sock_load_ocsp() on error path
MEDIUM: init: set default for fd_hard_limit via DEFAULT_MAXFD (take #2)
BUG/MEDIUM: init: fix fd_hard_limit default in compute_ideal_maxconn
William Lallemand (6):
REGTESTS: ssl: fix some regtests 'feature cmd' start condition
DOC: configuration: fix alphabetical order of bind options
DOC: configuration: more details about the master-worker mode
BUG/MINOR: jwt: don't try to load files with HMAC algorithm
BUG/MINOR: jwt: fix variable initialisation
DOC: configuration: issuers-chain-path not compatible with OCSP
Willy Tarreau (9):
MINOR: activity: make the memory profiling hash size configurable at
build time
BUG/MEDIUM: quic: fix possible exit from qc_check_dcid() without unlocking
BUG/MEDIUM: debug/cli: fix "show threads" crashing with low thread counts
BUG/MAJOR: mux-h2: force a hard error upon short read with pending error
DOC: config: improve the http-keep-alive section
MINOR: queue: add a function to check for TOCTOU after queueing
BUG/MEDIUM: queue: deal with a rare TOCTOU in assign_server_and_queue()
MINOR: mux-h2: try to clear DEM_MROOM and MUX_MFULL at more places
BUG/MAJOR: mux-h2: always clear MUX_MFULL and DEM_MROOM when clearing the
mbuf
--
Christopher Faulet
