Hi,
HAProxy 3.1-dev7 was released on 2024/09/05. It added 56 new commits
after version 3.1-dev6.
This version collects a non-negligible number of bug fixes (16) including
the important one affecting H2+zero-copy forwarding mentioned in the 3.0
and 2.9 announcements. The other bugs that were fixed concern frozen
connections, crashes, and an issue where rate counters could be bad and
remain bad for a while in case of time jumps.
Among the changes in this version, I'm seeing:
- support for applying a factor to byte rates in stick-tables. The issue
is that right now the rate values are stored on 32 bits and that this
limits measurable rates to about 34 Gbps. When this was added in 2010,
we wouldn't imagine that users would require more than this *per key*.
But now we're in 2024, the bwlim filter has become popular even on
large systems where high limits need to be set. It was figured that
the best way to address this specific (and admittedly rare) need is to
just apply a factor to the rate stored in the stick-table. This allows
the bwlim filter to be configured with lower values and reported rates
to still make sense. The "brates-factor" configured on the stick-table
supports a range of 1 to 1024. 1024 could imply counting kbytes per
second instead of bytes per second. 125 could be nice to count kbit/s
instead for example.
- the MPTCP support from Dorian, Anthony and Mathieu was finally merged.
MPTCP addresses can be explicitly specified using prefixes "mptcp@",
"mptcp4@", "mptcp6@". While writing this I'm realizing that I don't
know if we have an easy way to know if we're dealing with an MPTCP or a
regular TCP connection in order to feed stats. It could be interesting
to see in logs which connections were made using this and measure how
popular it is among clients. Hmmm maybe we can do that using one
listener each... More things to test :-)
- the init-state patch from Aaron was also merged. It finally allows
a server *not* to instantly turn on upon startup or when leaving
maintenance, but only after it succeeds its checks. There have been
requests for this for a very long time (issue 51!), and now 4 possible
behaviors can be configured per server. One use case I'm aware of is
when you set a server in maintenance to upgrade it, you usually don't
want it to instantly be reintegrated when marking it ready, instead
you usually prefer that it performs its checks. Now it will be
possible via the "init-state" server parameter.
- a few sample fetch functions (ssl_fc_supported_versions_bin and
ssl_fc_sigalgs_bin) were added to retrieve such information during
the SSL handshake that are otherwise not available later.
- the environment variables for CLF log formats were added
- the master process now restores its initial environment before
switching to wait mode. Indeed, previously it would re-exec itself
using the new variables parsed in the config. An example of problem
is if you do that in your config:
global
#setenv TRACE 1
.if defined(TRACE)
...
.endif
Then once you temporarily uncomment the "setenv", the TRACE variable
is set and passed to sub-processes, then if you comment the config
again it persists since it was inherited. This won't happen anymore
as we now keep a copy of the environment that is restored before the
execve(). Note that all such corner cases won't be an issue soon with
the ongoing change of the master/worker startup sequence anyway.
- support for "retry-on 429" was added.
- QUIC: the NEW_TOKEN frame support was added so as to always be able
to validate reachability of the client during 0-RTT. The problem was
reported by Michael Wedl, without this a client can change its address
between the connection establishment and a 0-RTT request, the last one
will be used and logged (as expected). The problem is that we don't
know if it's its real address and we need to validate it first. The
NEW_TOKEN allows to deliver an IP-based token to the client for use
later, and if the address changes, then we can use a regular RETRY
token. It was tested to be OK and now works fine even in the interop.
It was also backported to 3.0. Not sure how far it will go since it
relies on recent infrastructure updates.
The rest is extra debug info, cleanups and typo fixes, as well as CI
updates. I'm glad that we're finally merging long-standing changes that
have been discussed for several years.
If you're running any 3.1-dev, this one should be worth an update. It
contains essentially bug fixes (including the one against the risk of
crashing the H2 mux) and all other changes are really isolated, so except
by failing on a bugfix, the risk of regression from previous versions is
much lower than the risk to fix bugs.
Given that we now have some substance for another 3.0 and that Fred spent
a good time backporting and testing the QUIC stuff, I think we'll produce
3.0.5 next week (no promise as usual). 2.9 and 2.8 are almost ready for
another shot. 2.6 should not require too much work at first glance so it
should be within reach. 2.4 and 2.2 are older and will require more work
(if fixes are needed at all, which is not even granted).
We still have roughly 3 months before the release, that's fine but that's
not a lot. As a reminder, September is usually busy for a number of us
(meetings, conferences, even releases of enterprise stuff for some), so I
expect less regular progress in the coming weeks and less time for reviews
as well. This means that sensitive stuff really needs to be prioritized so
that we still have sufficient testing time (and that includes my ongoing
work on H2 rx window). We're having a pretty nice 3.1 for now, which seems
to be in a better shape than 3.0 when it was released, I would hate to
degrade it in the last weeks, so please all keep this in mind.
Please find the usual URLs below :
Site index : https://www.haproxy.org/
Documentation : https://docs.haproxy.org/
Wiki : https://github.com/haproxy/wiki/wiki
Discourse : https://discourse.haproxy.org/
Slack channel : https://slack.haproxy.org/
Issue tracker : https://github.com/haproxy/haproxy/issues
Sources : https://www.haproxy.org/download/3.1/src/
Git repository : https://git.haproxy.org/git/haproxy.git/
Git Web browsing : https://git.haproxy.org/?p=haproxy.git
Changelog : https://www.haproxy.org/download/3.1/src/CHANGELOG
Dataplane API :
https://github.com/haproxytech/dataplaneapi/releases/latest
Pending bugs : https://www.haproxy.org/l/pending-bugs
Reviewed bugs : https://www.haproxy.org/l/reviewed-bugs
Code reports : https://www.haproxy.org/l/code-reports
Latest builds : https://www.haproxy.org/l/dev-packages
Willy
---
Complete changelog :
Aaron Kuehler (1):
MEDIUM: server: add init-state
Amaury Denoyelle (2):
MINOR: mux-quic: add buf_in_flight to QCC debug infos
MINOR: mux-quic: correct qcc_bufwnd_full() documentation
Aperence (4):
MINOR: tools: extend str2sa_range to add an alt parameter
MINOR: server: add a alt_proto field for server
MEDIUM: sock: use protocol when creating socket
MEDIUM: protocol: add MPTCP per address support
Christopher Faulet (10):
MINOR: proxy: Add support of 429-Too-Many-Requests in retry-on status
BUG/MEDIUM: mux-h2: Set ES flag when necessary on 0-copy data forwarding
BUG/MEDIUM: stream: Prevent mux upgrades if client connection is no
longer ready
BUG/MINIR: proxy: Match on 429 status when trying to perform a L7 retry
MEDIUM: stick-table: Add support of a factor for IN/OUT bytes rates
MEDIUM: bwlim: Use a read-lock on the sticky session to apply a shared
limit
BUG/MEDIUM: mux-pt: Never fully close the connection on shutdown
BUG/MEDIUM: cli: Always release back endpoint between two commands on the
mcli
BUG/MEDIUM: mux-h1: Properly handle empty message when an error is
triggered
BUG/MEDIUM: mux-pt: Fix condition to perform a shutdown for writes in
mux_pt_shut()
Frederic Lecaille (12):
BUILD: quic: 32bits build broken by wrong integer conversions for printf()
MINOR: tools: Implement ipaddrcpy().
MINOR: quic: Implement quic_tls_derive_token_secret().
MINOR: quic: Token for future connections implementation.
BUG/MINOR: quic: Missing incrementation in NEW_TOKEN frame builder
MINOR: quic: Modify NEW_TOKEN frame structure (qf_new_token struct)
MINOR: quic: Implement qc_ssl_eary_data_accepted().
MINOR: quic: Add trace for QUIC_EV_CONN_IO_CB event.
BUG/MEDIUM: quic: always validate sender address on 0-RTT
BUG/MINOR: quic: Crash from trace dumping SSL eary data status (AWS-LC)
BUG/MINOR: quic: unexploited retransmission cases for Initial pktns.
BUG/MINOR: Crash on O-RTT RX packet after dropping Initial pktns
Ilya Shipitsin (7):
DEV: coccinelle: add a test to detect unchecked malloc()
DEV: coccinelle: add a test to detect unchecked calloc()
CI: QUIC Interop AWS-LC: enable ngtcp2 client
CI: fix missing comma introduced in
956839c0f68a7722acc586ecd91ffefad2ccb303
CI: QUIC Interop: do not run bandwidth measurement tests
CI: QUIC Interop: use different artifact names for uploading logs
CLEANUP: assorted typo fixes in the code and comments
Nathan Wehrman (1):
MINOR: config: Created env variables for http and tcp clf formats
Nicolas CARPi (2):
CLEANUP: haproxy: fix typos in code comment
CLEANUP: mqtt: fix typo in MQTT_REMAINING_LENGHT_MAX_SIZE
Valentine Krasnobaeva (3):
MINOR: tools: add helpers to backup/clean/restore env
MINOR: mworker: restore initial env before wait mode
BUG/MINOR: haproxy: free init_env in deinit only if allocated
William Lallemand (6):
BUILD: tools: environ is not defined in OS X and BSD
CLEANUP: ssl: cleanup the clienthello capture
MEDIUM: ssl: capture the supported_versions extension from Client Hello
MEDIUM: ssl/sample: add ssl_fc_supported_versions_bin sample fetch
MEDIUM: ssl: capture the signature_algorithms extension from Client Hello
MEDIUM: ssl/sample: add ssl_fc_sigalgs_bin sample fetch
Willy Tarreau (8):
BUILD: quic: fix build errors on FreeBSD since recent GSO changes
MINOR: mux-h2: try to clear DEM_MROOM and MUX_MFULL at more places
BUG/MAJOR: mux-h2: always clear MUX_MFULL and DEM_MROOM when clearing the
mbuf
BUG/MINOR: mux-spop: always clear MUX_MFULL and DEM_MROOM when clearing
the mbuf
DEV: patchbot: count the number of backported/non-backported patches
DEV: patchbot: add direct links to show only specific categories
DEV: patchbot: detect commit IDs starting with 7 chars
BUG/MEDIUM: clock: also update the date offset on time jumps
---
