Hi,
HAProxy 3.4-dev9 was released on 2026/04/15. It added 162 new commits
after version 3.4-dev8.
46 bugs were fixed again in various areas, a few of them possibly causing
the process to crash under certain conditions. Some of these fixes will
need to be backported, we'll probably emit new stable releases next week.
The crash I noticed two weeks ago on haproxy.org happened again a few days
ago on 3.4-dev8, and again no core (though we know it was a memcpy). We
now have good reasons to think it could be caused by a recently introduced
bug in HTX that was fixed in this -dev9, so the next two weeks will tell
us whether we're right or not.
Now let's go to the features, as there are some good news again.
- OpenTelemetry: the looooong awaited filter was now merged! I'd just
say: please have a look at addons/otel/README as there's a ton of
doc and I'm unable to summarize what I glanced over there.
It's still in experimental status, because for now it requires the
nasty "insecure-fork-wanted" that nobody wants to set on an exposed
node for obvious reasons. But Miroslav is still trying hard to figure
how we can work around runtime thread creation in the OTEL SDK to avoid
this major pain. Being able to start them at boot would be the minimum
required to drop the experimental status and a first step in the right
direction. Ideally getting rid of the threads entirely to instead use
asynchronous tasks like everywhere else in the process would avoid
needlessly degrading the quality of service, but I can easily grasp
how non-trivial a task it can be with such monster libraries.
To all those having been waiting for it, please test it and provide
feedback before 3.4-release, especially on the compilation and the
configuration fronts, because if there's something that you feel
unconvenient, it may still be addressed before the release, it won't
after. There may be choices to be made for distro maintainers by the
way, as I understood that the chain of dependencies is not simple. So
if you spot showstoppers, that's pretty useful to know early.
- ACME: the dns-persist-01 challenge is now implemented. This is the
piece that has been keeping me away from LetsEncrypt for a long time,
because previously, either you used the http-01 challenge and were
limited to explicit host names, or you used dns-01 to benefit from
wildcards, but you had to basically give full access to your DNS zone
to the tool. With dns-persist-01, you have the best of both worlds,
you create an entry in your DNS once, and you're done. The only thing
is that for now it's only supported by staging LetsEncrypt servers,
so we'll have to wait a bit to deploy it on public servers.
- checks: more updates to better refine the choice of protocols used by
checks. The previous batch in -dev8 was not sufficient and left a few
corner cases behind. So if you faced some server check configuration
issues, these are hopefully solved now. Also, health checks now support
sending a unique-id, which can be convenient if your application want
one on each incoming request.
- filters: the new "filter-sequence" directive allows to specify, for
each direction (request/response), the sequence of filters to be
called. This is a prerequisite for the decompression filter to come,
though it could possibly find its use cases as-is right now.
- QMUX: the QMUX (QUIC over TCP) implementation was updated to the
new IETF draft-01 which now uses records.
- QUIC: new settings were implemented to support an enforcement of
the maximum number of total streams over a connection
(tune.quic.fe.stream-max-total), like we have with H2. This may be used
to force clients to periodically renew a connection (might be useful in
order to mitigate certain classes of attacks in the future for example).
- threads: since the introduction of CPU topology in 3.2, if nbthreads
was forced, it could be silently ignored and limited to the max number
of threads per group (typically 64). It wasn't much visible by then but
now with the max-threads-per-group directive, it is obvious. This was
changed so that the required number of thread groups is now created to
match this, even if policies are not used. It will be backported to 3.3
as automatic binding really started there, and might fix the number of
threads on certain large configs that were manually tuned (configs
having more than 64 threads and no thread-groups are currently under
powered). I'm undecided for 3.2. In the next -dev, we'll reduce the
default max-threads-per-group so that machines with large unified
caches are not penalized by having too many threads per group. Tests
show that 8 or 16 are pretty good, we'll see.
- Lua: a new directive "tune.lua.openlibs" allows to indicate which of
the default Lua libs should be loaded at boot. It turns out that we're
absolutely not forced to support os.xxx() for example, so it is possible
to host some Lua much more securely by removing everything non-essential.
- minor stuff, such as new sample fetch functions to report CPU usage and
number of the current thread group, number of thread groups in "show info",
TLS 1.2 values support in HAPROXY_KEYLOG_XX_LOG_FMT
- there's also a significant number of updates for the CI (thanks for this),
and cleanups all over the code.
With the merge of OpenTelemetry, I'd like us to stick to the originally
planned deprecation schedule of OpenTracing. It has been deprecated for
the last 4 years, I doubt anyone is really still using it, but since it
was still available in 3.3, it must remain available in 3.4. The goal
would be to proceed like this:
- mark it as deprecated in 3.4 and emit a deprecation warning at boot
- mark it as soon deprecated in 3.3 and emit a deprecation *notice* at
boot (i.e. never fatal)
- remove it from 3.5.
This would catch up with the documented plans in the Wiki (even if the
initial plans were for a removal in 3.3). I'm interested in feedback
here. If some say "we will still need it for many years and beyond 3.4",
maybe it can stay. If instead someone says "don't bother with it, you
can't build it anymore", maybe we can directly remove it. But my
preference would be for the usual smooth approach.
Please find the usual URLs below :
Site index : https://www.haproxy.org/
Documentation : https://docs.haproxy.org/
Wiki : https://github.com/haproxy/wiki/wiki
Discourse : https://discourse.haproxy.org/
Slack channel : https://slack.haproxy.org/
Issue tracker : https://github.com/haproxy/haproxy/issues
Q&A from devs : https://github.com/orgs/haproxy/discussions
Sources : https://www.haproxy.org/download/3.4/src/
Git repository : https://git.haproxy.org/git/haproxy.git/
Git Web browsing : https://git.haproxy.org/?p=haproxy.git
Changelog : https://www.haproxy.org/download/3.4/src/CHANGELOG
Dataplane API :
https://github.com/haproxytech/dataplaneapi/releases/latest
Pending bugs : https://www.haproxy.org/l/pending-bugs
Reviewed bugs : https://www.haproxy.org/l/reviewed-bugs
Code reports : https://www.haproxy.org/l/code-reports
Latest builds : https://www.haproxy.org/l/dev-packages
Willy
---
Complete changelog :
Amaury Denoyelle (21):
BUG/MINOR: mux-quic: fix potential NULL deref on qcc_release()
BUG/MINOR: quic: increment pos pointer on QMux transport params parsing
MINOR: xprt_qstrm: implement Rx buffering
MINOR: xprt_qstrm/mux-quic: handle extra QMux frames after params
MINOR: xprt_qstrm: implement Tx buffering
MINOR: xprt_qstrm: handle connection errors
MEDIUM: mux-quic: implement QMux record parsing
MEDIUM: xprt_qstrm: implement QMux record parsing
MEDIUM: mux-quic/xprt_qstrm: implement QMux record emission
DOC: update draft link for QMux protocol
BUG/MINOR: do not crash on QMux reception of BLOCKED frames
BUG/MINOR: mux_quic: prevent QMux crash on qcc_io_send() error path
BUG/MINOR: xprt_qstrm: do not parse record length on read again
MINOR: mux_quic: remove duplicate QMux local transport params
BUG/MINOR: quic: do not use hardcoded values in QMux TP frame builder
MINOR: mux-quic: improve documentation for qcs_attach_sc()
MINOR: mux-quic: reorganize code for app init/shutdown
MINOR: mux-quic: perform app init in case of early shutdown
MEDIUM: quic: implement fe.stream.max-total
MINOR: mux-quic: close connection when reaching max-total streams
REGTESTS: add QUIC test for max-total streams setting
Aurelien DARRAGON (5):
DOC: config: fix ambiguous info in log-steps directive description
MINOR: filters: add filter name to flt_conf struct
MEDIUM: filters: add "filter-sequence" directive
REGTESTS: add a test for "filter-sequence" directive
BUG/MINOR: counters: fix unexpected 127 char GUID truncation for
shm-stats-file objects
Christopher Faulet (16):
BUG/MEDIUM: haterm: Move all init functions of haterm in haterm_init.c
BUG/MEDIUM: mux-h1: Disable 0-copy forwarding when draining the request
DOC: config: Fix two typos in the server param "healthcheck" description
MINOR: tcpcheck: Reject unknown keyword during parsing of healthcheck
section
BUG/MEDIUM: tcpcheck/server: Fix parsing of healthcheck param for dynamic
servers
BUG/MEDIUM: tcpcheck: Properly retrieve tcpcheck type to install the best
mux
BUG/MEDIUM: connection: Wake the stconn on error when failing to create
mux
BUG/MINOR: haterm: Return the good start-line for 100-continue interim
message
BUG/MEDIUM: samples: Fix handling of SMP_T_METH samples
Revert "BUG/MEDIUM: haterm: Move all init functions of haterm in
haterm_init.c"
BUG/MEDIUM: haterm: Properly initialize the splicing support for haterm
BUG/MEDIUM: cli: Properly handle too big payload on a command line
REGTESTS: Never reuse server connection in reg-tests/jwt/jwt_decrypt.vtc
MINOR: htx: Add helper function to get type and size from the block info
field
BUG/MEDIUM: htx: Properly handle block modification during defragmentation
BUG/MEDIUM: htx: Don't count delta twice when block value is replaced
Egor Shestakov (3):
MINOR: tools: memvprintf(): remove <out> check that always true
MINOR: errors: remove excessive errmsg checks
BUG/MINOR: acme: don't pass NULL into format string
Emeric Brun (1):
BUG/MEDIUM: peers: trash of expired entries delayed after fullresync
Greg Kroah-Hartman (16):
BUG/MEDIUM: payload: validate SNI name_len in req.ssl_sni
BUG/MEDIUM: jwe: fix NULL deref crash with empty CEK and non-dir alg
BUG/MEDIUM: jwt: fix heap overflow in ECDSA signature DER conversion
BUG/MEDIUM: jwe: fix memory leak in jwt_decrypt_secret with var argument
BUG: hlua: fix stack overflow in httpclient headers conversion
BUG/MINOR: hlua: fix stack overflow in httpclient headers conversion
BUG/MINOR: hlua: fix format-string vulnerability in Patref error path
BUG/MEDIUM: chunk: fix typo allocating small trash with bufsize_large
BUG/MEDIUM: chunk: fix infinite loop in get_larger_trash_chunk()
BUG/MINOR: peers: fix OOB heap write in dictionary cache update
BUG/MINOR: resolvers: fix memory leak on AAAA additional records
BUG/MINOR: spoe: fix pointer arithmetic overflow in spoe_decode_buffer()
BUG/MINOR: http-act: validate decoded lengths in *-headers-bin
BUG/MINOR: sample: fix info leak in regsub when exp_replace fails
BUG/MEDIUM: mux-fcgi: prevent record-length truncation with large bufsize
BUG/MINOR: hlua: fix use-after-free of HTTP reason string
Ilia Shipitsin (1):
CI: build WolfSSL job with asan enabled
Mia Kanashi (1):
MEDIUM: acme: implement dns-persist-01 challenge
Miroslav Zagorac (33):
MEDIUM: otel: added OpenTelemetry filter skeleton
MEDIUM: otel: added configuration and utility layer
MEDIUM: otel: added configuration parser and event model
MEDIUM: otel: added post-parse configuration check
MEDIUM: otel: added memory pool and runtime scope layer
MEDIUM: otel: implemented filter callbacks and event dispatcher
MEDIUM: otel: wired OTel C wrapper library integration
MEDIUM: otel: implemented scope execution and span management
MEDIUM: otel: added context propagation via carrier interfaces
MEDIUM: otel: added HTTP header operations for context propagation
MEDIUM: otel: added HAProxy variable storage for context propagation
MINOR: otel: added prefix-based variable scanning
MEDIUM: otel: added CLI commands for runtime filter management
MEDIUM: otel: added group action for rule-based scope execution
MINOR: otel: added log-format support to the sample parser and runtime
MINOR: otel: test: added test and benchmark suite for the OTel filter
MINOR: otel: added span link support
MINOR: otel: added metrics instrument support
MINOR: otel: added log-record signal support
MINOR: otel: test: added full-event test config
DOC: otel: added documentation
DOC: otel: test: added test README-* files
DOC: otel: test: added speed test guide and benchmark results
DOC: otel: added cross-cutting design patterns document
MINOR: otel: added flt_otel_sample_eval and exposed flt_otel_sample_add_kv
MINOR: otel: changed log-record attr to use sample expressions
MINOR: otel: changed instrument attr to use sample expressions
DOC: otel: added README.md overview document
CLEANUP: ot: use the item API for the variables trees
BUG/MINOR: ot: removed dead code in flt_ot_parse_cfg_str()
BUG/MINOR: ot: fixed wrong NULL check in flt_ot_parse_cfg_group()
BUILD: ot: removed explicit include path when building opentracing filter
MINOR: ot: renamed the variable dbg_indent_level to
flt_ot_dbg_indent_level
Olivier Houchard (4):
MINOR: servers: The right parameter for idle-pool.shared is "full"
MEDIUM: connections: Really enforce mux protocol requirements
MINOR: tools: Implement net_addr_type_is_quic()
MEDIUM: check: Revamp the way the protocol and xprt are determined
Tim Duesterhus (29):
CI: Drop obsolete `packages: write` permission from `quic-interop-*.yml`
CI: Consistently add a top-level `permissions` definition to GHA workflows
CI: Wrap all `if:` conditions in `${{ }}`
CI: Fix regular expression escaping in matrix.py
CI: Update to actions/checkout@v6
CI: Simplify version extraction with `haproxy -vq`
CI: Merge `aws-lc.yml` and `aws-lc-fips.yml` into `aws-lc.yml`
CI: Merge `aws-lc-template.yml` into `aws-lc.yml`
CI: Consistently set up VTest with `./.github/actions/setup-vtest`
BUG/MINOR: log: Fix error message when using unavailable fetch in logfmt
CLEANUP: log: Return `size_t` from `sess_build_logline_orig()`
CLEANUP: stream: Explain the two-step initialization in
`stream_generate_unique_id()`
CLEANUP: stream: Reduce duplication in `stream_generate_unique_id()`
CLEANUP: http_fetch: Use local `unique_id` variable in
`smp_fetch_uniqueid()`
CLEANUP: Make `lf_expr` parameter of `sess_build_logline_orig()` const
MINOR: Add `generate_unique_id()` helper
MINOR: Allow inlining of `stream_generate_unique_id()`
CLEANUP: log: Stop touching `struct stream` internals for `%ID`
MINOR: check: Support generating a `unique_id` for checks
MINOR: http_fetch: Add support for checks to `unique-id` fetch
CI: Remove obsolete steps from musl.yml
CI: Use `sh` in `actions/setup-vtest/action.yml`
CI: Sync musl.yml with vtest.yml
CI: Integrate Musl build into vtest.yml
CI: Use `case()` function
CI: Generate vtest.yml matrix on `ubuntu-slim`
CI: Run contrib.yml on `ubuntu-slim`
CI: Use `matrix:` in contrib.yml
CI: Build `dev/haring/` as part of contrib.yml
William Lallemand (21):
Revert "CLEANUP: tcpcheck: Don't needlessly expose proxy_parse_tcpcheck()"
MINOR: tcpcheck: reintroduce proxy_parse_tcpcheck() symbol
CI: VTest build with git clone + cache
CI: github: update to cache@v5
Revert "BUG: hlua: fix stack overflow in httpclient headers conversion"
CI: github: fix vtest path to allow correct caching
CI: github: add the architecture to the cache key for vtest2
MINOR: lua: add tune.lua.openlibs to restrict loaded Lua standard
libraries
REGTESTS: lua: add tune.lua.openlibs to all Lua reg-tests
CI: github: add bash to the musl job
MINOR: acme: extend resolver-based DNS pre-check to dns-persist-01
DOC: configuration: document dns-persist-01 challenge type and options
BUG/MINOR: acme: read the wildcard flag from the authorization response
MINOR: acme: display the type of challenge in ACME_INITIAL_DELAY
MINOR: mjson: reintroduce mjson_next()
MINOR: ssl: add TLS 1.2 values in HAPROXY_KEYLOG_XX_LOG_FMT
EXAMPLES: ssl: keylog entries are greater than 1024
DOC: remove the alpine/musl status job image
MINOR: acme: opportunistic DNS check for dns-persist-01 to skip
challenge-ready steps
BUG/MINOR: acme: fix fallback state after failed initial DNS check
CLEANUP: acme: no need to reset ctx state and http_state before nextreq
Willy Tarreau (11):
BUG/MINOR: http-act: fix a typo in the "pause" action error message
BUG/MAJOR: slz: always make sure to limit fixed output to less than worst
case literals
BUG/MINOR: haterm: preserve the pipe size margin for splicing
BUG/MINOR: haterm: don't apply the default pipe size margin twice
BUILD: Makefile: don't forget to also delete haterm on make clean
MINOR: stats: report the number of thread groups in "show info"
CLEANUP: sample: fix the comment regarding the range of the thread sample
fetch
MINOR: sample: return the number of the current thread group
MINOR: sample: add new sample fetch functions reporting current CPU usage
MEDIUM: threads: start threads by groups
BUG/MINOR: threads: properly set the number of tgroups when non using
policy
---
