Hi,
HAProxy 3.1-dev14 was released on 2024/11/21. It added 132 new commits
after version 3.1-dev13.
OK that's basically the end of the cleanups, fixes, tests and last minute
merges. So far so good.
- various reg-tests were cleaned up and fixed; they now all run at -dW
meaning that we can also catch config warnings.
- a nasty bug in the watchdog was detected thanks to the shorter warning
time and it could kill earlier than expected if a task was stuck twice
for a short time with a pause in between. It could also be sped up by
issuing "show threads" in case the last time slot expired at the exact
moment a thread was being dumped (something I already experienced once
or twice in the past without understanding).
- some fixes for memory profiling which used to report incorrect aggregate
values for the per-DSO summary. Also we can now also monitor strdup()
calls as well as a few non-portable ones (strndup, valloc, memalign etc).
This should fix some rare occurrences were the delta appeared negative
(more free than alloc).
- for the master-worker mode, expose-deprecated-directives now correctly
silences the warning about the "programs" section being deprecated. Also,
since the code related to the systemd mode does no longer rely on an
external lib, it's now always enabled and the USE_SYSTEMD build option
was now removed. This also simplifies the tests since the latest vtest
can provide and monitor the systemd socket and this allows to test it on
all platforms (note that for a yet unknown reason, it's being constantly
failing on OSX so these tests were not enabled there).
- a use-after-free on startup when using "log-tag" was fixed, and the use
of the various HAPROXY_* variables was clarified, uniformized between
master and worker processes, and they were finely documented so that it's
now easier to know which one can be used/modified/exported etc.
- the "chroot" command now benefits from a parse-time test that can report
inconsistent directories or permissions in a way that is compatible with
-c. However since -c is supposed to work from basically anywhere and with
minimal permissions, this is only reported as a diag warning (-dD), which
allows those who care about this to see the report without annoying other
ones.
- the traditional makefile reordering was done to save a few seconds on
multi-core systems. Usually this is a sign we're getting closer ;-)
Also, ERR=1 will now also catch the makefile's warnings (e.g. misspelled
USE_* variables or outdated flags being ignored).
- the previously suggested warning on unit-less small timeouts was finally
applied to those with only two digits (e.g. "timeout client 30"), because
such small values tend to raise implicit expectations that these are
expressed in seconds. I've had to deal at least with this twice over the
last year. The warning suggests how to change that ("timeout client 30s").
I've found one occurrence of these only once in my test configs so only
those at risk should notice it.
- the HTTP/1 mux can now report a 414 or 431 when the request is too large
to fit in a buffer or the URI is too long.
- the "when" converter now supports "acl,<acl_name>" as arguments, to
condition passing the input based on more dynamic rules. The first use
case that comes to mind and was suggested by Tristan is to only log
details of requests resulting in long transfer times. But others might
work as well (checking certain statuses, termination codes etc).
- the tasklet_wakeup() family of functions now support an optional
argument to pass a flag like for the tasks. It's not used yet but will
maintain better uniformity with future code for backports and might be
necessary later to backport some fixes.
- the stats-file now ignores comment lines starting with '//' like in the
regtest examples.
- the "localpeer" keyword is now processed in discovery mode so that both
master and worker agree on it, and more importantly that config elements
relying on the HAPROXY_LOCALPEER variable continue to work like in 3.0.
- a number of config directives that expect a size (tune.bufsize, ring sizes
etc) now accept a suffix. Previously the suffix was silently ignored so
that when writing, say, "tune.pipesize 512k", one would end up with a
configured pipe of 512 only! There are still many places with such old
atol() code that is progressively being replaced, but it takes time.
- "show env" on the master CLI no longer requires debug mode.
- warnings and indications about misplaced TCP rules will now report
the whole directive name, not just "tcp-request" which was ambiguous.
- redirect rules now support "keep-query" to reuse the original request's
query string, and "set-cookie-fmt" to pass a cookie while redirecting.
Also, the "query()" sample fetch function now takes an optional argument
"with_qm" to request that the question mark is preserved if it exists
(this simplifies writing redirections).
- "show sess" now has a "show-uri" option to show the captured URI.
- the "core.set_map()" lookup in Lua is now more efficient by no longer
performing the lookup twice (that was already done everywhere else in
2.9 and that one was overlooked).
- agent-checks now support passing an absolute weight.
- the logged server status is now correct after an L7 retry. Previously
it used to report only the first code that triggered the retry.
- an interesting overlook in the H2 mux would possibly cause client-side
failures when too many headers came from the server over H2 as well,
because the receive side was more permissive than the send side (it had
a 100% margin on the number of headers to be able to swallow split
cookie headers, but the send side would choke at the default limit).
Now the real limit is controlled at the moment the headers are deduped
and indexed so tha both sides have the same vision.
- and the nice part (still experimental but will significantly help
backport fixes and even continue to progress) concerns QUIC. The
pacing code was finally merged in experimental state (opt-in, not
active by default), and the long-awaited BBR congestion controller
which needs pacing was also merged. Interestingly the pacing when
used tends to magnify certain races in the code, which we could
confirm were already there without it. At the moment, enabling it
significantly reduces losses and shows much higher bit rates (up to
16x on one of my machine when the client runs on a small core), but
also shows some variations in the bit rate that are caused by some
delays that we're still trying to figure and seem to be present
without the pacing as well. You may want to experiment a bit with
it but be aware that this is experimental, and it consumes much
more CPU (having to make sub-millisecond pauses is expensive).
However it already works so much better that I suspect that we'll
gather some feedback allowing to polish the rough edges. And
according to Fred, BBR now delivers performance that is on par with
TCP+BBR. I guess that 3.2 will have a much faster QUIC stack and
that most likely after a few stable releases, 3.1 as well. Time
will tell.
- and the rest is essentially code cleanups and doc updates (14
commits just for this one).
Many build+run+vtest checks were run in using configs options on
different systems, versions, compilers and libcs (linux + glibc/musl
from gcc-4.8 to 13.2 on x86_64, armv7, armv8, FreeBSD x86-64 with
clang, OpenBSD 7.5 on mips64, more to come). BTW I've just noticed
a build warning in activity.c on gcc-4.8 when using memory profiling
while I was typing, I'll fix that tomorrow.
Overall that was lots of nice small goodies, which again will take me
a while to summarize in the annouce! Given how everything's getting
good, barring any problematic report till then, I intend to release
this next Tuesday. That doesn't mean that you should skip this one,
because if nobody tests, the bugs of this last dev version will be
those of the first stable one!
Thanks for continuing to test and report problems!
Please find the usual URLs below :
Site index : https://www.haproxy.org/
Documentation : https://docs.haproxy.org/
Wiki : https://github.com/haproxy/wiki/wiki
Discourse : https://discourse.haproxy.org/
Slack channel : https://slack.haproxy.org/
Issue tracker : https://github.com/haproxy/haproxy/issues
Sources : https://www.haproxy.org/download/3.1/src/
Git repository : https://git.haproxy.org/git/haproxy.git/
Git Web browsing : https://git.haproxy.org/?p=haproxy.git
Changelog : https://www.haproxy.org/download/3.1/src/CHANGELOG
Dataplane API :
https://github.com/haproxytech/dataplaneapi/releases/latest
Pending bugs : https://www.haproxy.org/l/pending-bugs
Reviewed bugs : https://www.haproxy.org/l/reviewed-bugs
Code reports : https://www.haproxy.org/l/code-reports
Latest builds : https://www.haproxy.org/l/dev-packages
Willy
---
Complete changelog :
Amaury Denoyelle (21):
DOC: quic: rename max-window-size as with default prefix
MINOR: mux-quic: add missing values for show flags
MINOR: quic: simplify qc_prep_pkts() exit path
MINOR: quic: support a max number of built packet per send iteration
MINOR: quic: extend qc_send_mux() return type with a dedicated enum
MINOR: quic: define quic_pacing module
MINOR: quic/pacing: implement quic_pacer engine
MINOR: quic/pacing: support pacing emission on quic_conn layer
MINOR: quic/pacing: add burst support
MINOR: mux-quic: define a tx STREAM frame list member
MINOR: mux-quic: encapsulate QCC tasklet wakeup
MAJOR: mux-quic: support pacing emission
MINOR: quic: use dynamic cc_algo on bind_conf
MINOR: quic: extend quic-cc-algo optional parameters
MEDIUM: quic: define cubic-pacing congestion algorithm
MINOR: mux_quic/pacing: display pacing info on show quic
BUG/MINOR: cfgparse-quic: fix renaming of max-window-size
BUG/MINOR: cfgparse-quic: fix bbr initialization
MINOR: cfgparse-quic: activate pacing only via burst argument
BUG/MINOR: cfgparse-quic: fix warning for cc-aglo with 0 burst
MINOR: quic: support pacing for newreno and nocc
Aurelien DARRAGON (2):
DOC: lua: fix yield-dependent methods expected contexts
OPTION: map/hlua: make core.set_map() lookup more efficient
Christopher Faulet (24):
MINOR: http-ana: Add option to keep query-string on a localtion-based
redirect
MINOR: http-ana: Add support for "set-cookie-fmt" option to redirect rules
MINOR: agent-check: Be able to set absolute weight via an agent
MINOR: stream: Add an option to "show sess" command to dump the captured
URI
DOC: config: A a space before ':' for {bs,fs}.aborted and {bs,fs}.rst_code
DOC: config: Fix a typo in "1.3.1. The Request line"
MINOR: http: Add support for HTTP 414/431 status codes
DEV: phash: Update 414 and 431 status codes to phash
MINIR: mux-h1: Return 414 or 431 when appropriate
BUG/MINOR: http_ana: Report -1 for %Tr for invalid response only
DOC: config: Slightly improve the %Tr documentation
DOC: config: Move wait_end in section about internal samples
DOC: config: Move fs.* and bs.* in section about L5 samples
BUG/MINOR: http-ana: Adjust the server status before the L7 retries
MINOR: http-fetch: Add an option to 'query" to get the QS with the '?'
BUG/MEDIUM: mux-h2: Increase max number of headers when encoding HEADERS
frames
BUG/MEDIUM: mux-h2: Check the number of headers in HEADERS frame after
decoding
BUG/MEDIUM: h3: Properly limit the number of headers received
BUG/MEDIUM: h3: Increase max number of headers when sending headers
DOC: config: Improve documentation of tune.http.maxhdr directive
DOC: management: Clearly state "show errors" only reports malformed H1
messages
MINOR: config: Improve warnings on misplaced rules by adding an optional
arg
CLEANUP: cfgparse: Add direction in functions name that warn on misplaced
rules
MINOR: cfgparse: Emit a warning for misplaced "tcp-response content" rules
Frederic Lecaille (11):
MINOR: quic: Add the congestion window initial value to QUIC path
MINOR: window_filter: Implement windowed filter (only max)
MINOR: quic: implement delivery rate sampling algorithm
MINOR: quic: implement BBR congestion control algorithm for QUIC
MINOR: quic: quic_cc modifications to support BBR
MINOR: quic: quic_loss modifications to support BBR
MINOR: quic: RX part modifications to support BBR
MINOR: quic: TX part modifications to support BBR.
MINOR: quic: add "bbr" new "quic-cc-algo" option
MINOR: quic: Useless rate sample member initialization
BUG/MINOR: quic: Missing application limitations tracking for BBR
Valentine Krasnobaeva (14):
MINOR: cfgparse-global: parse options to allow non std keywords in
discovery mode
BUG/MINOR: mworker-prog: don't warn about deprecated section with
expose-deprecated-directives
MINOR: cli: make "show env" accessible via master CLI without enabling
debug
MINOR: config: show HAPROXY_BRANCH in "show env" output
MINOR: startup: set HAPROXY_LOCALPEER only once
DOC: configuration: update "Environment variables" chapter
MINOR: cfgparse-global: add cfg_parse_global_chroot
MINOR: cfgparse-global: add more checks for "chroot" argument
BUG/MINOR: startup: fix UAF when set the default for log_tag
MINOR: capabilities: rename program_name argument to progname
MINOR: startup: use global progname variable
MINOR: cfgparse-global: add cfg_parse_global_localpeer
BUG/MINOR: config: allow to check HAPROXY_LOCALPEER in config
BUG/MINOR: startup: init_early: remove obsolete comment
William Lallemand (9):
MINOR: stats-file: add the filename in the warning
MEDIUM: stats-file: explicitely ignore comments starting by //
MEDIUM: stats-file: silently ignore be/fe mistmatch
REGTESTS: use -dW by default on every reg-tests
MEDIUM: mworker: remove USE_SYSTEMD requirement for -Ws
CI: vtest: temporarily build from the sd-notify PR
MINOR: systemd: replace SOCK_CLOEXEC by fcntl call to FD_CLOEXEC
REGTESTS: switch to -Ws for master-worker reg-tests
REGTESTS: disable temporarly mworker test on OSX
Willy Tarreau (51):
MINOR: acl: export find_acl_default()
MINOR: sample: extend the "when" converter to support an ACL
MINOR: cfgparse: parse tune.{rcvbuf,sndbuf}.{client,server} as sizes
MINOR: cfgparse: parse tune.{rcvbuf,sndbuf}.{frontend,backend} as sizes
MINOR: cfgparse: parse tune.pipesize as a size
MINOR: cfgparse: parse tune.recv_enough as a size
MINOR: cfgparse: parse tune.bufsize as a size
MINOR: cfgparse: parse tune.bufsize.small as a size
REGTESTS: silence the "log format ignored" warnings
REGTESTS: silence warning "previous 'http-response' action is final"
REGTESTS: make the unit explicit for very short timeouts
REGTESTS: silence warnings about content-type being ignored
REGTESTS: remove a duplicate "option httpslog" in the defaults section
REGTESTS: silence warning "L6 sample fetches ignored" in cond_set_var
REGTESTS: add missing timeouts to 30 tests
REGTESTS: only use tune.ssl.default-dh-param when not using AWS-LC
REGTESTS: enable -dW on almost all tests to fail on warnings
MEDIUM: config: warn on unitless timeouts < 100 ms
MINOR: tools: make parse_size_err() support 32/64 bits
MINOR: ring: support unit suffixes in the size
DOC: sched: add missing scheduler API documentation for
tasklet_wakeup_after()
DOC: sched: document the missing TASK_F_UEVT* flags
CLEANUP: tinfo: move sched_*_date/*_mono_time to the thread-local area
MINOR: stream: don't update s->lat_time when the wakeup date is not set
MINOR: tinfo/clock: turn sched_call_date to 64-bits
MINOR: sched: add TASK_F_WANTS_TIME to make the scheduler update the call
date
MINOR: tools: add new macro DEFZERO to provide a default zero argument
MINOR: tasklet: make the low-level tasklet API take a flag
MINOR: tasklet: support an optional set of wakeup flags to
tasklet_wakeup_on()
DOC: configuration: explain the rules regarding spaces in arguments
DOC: configuration: explain quotes and spaces in conditional blocks
DOC: configuration: wrap long line for "strstr()" conditional expression
BUILD: makefile: make ERR apply to build options as well
DOC: config: indent the list of environment variables
BUILD: makefile: build flags.c before haproxy to speed up the build
BUILD: makefile: reorder object files by build time
BUG/MEDIUM: debug: don't set the STUCK flag from debug_handler()
BUG/MEDIUM: wdt: fix the stuck detection for warnings
BUG/MINOR: activity/memprofile: reinitialize the free calls on DSO summary
MINOR: activity/memprofile: offer a function to unregister stale info
BUG/MEDIUM: pools/memprofile: always clean stale pool info on
pool_destroy()
MINOR: activity: better report nil than ffff in unknown callers
CLEANUP: activity: better use a mask to tests freeing methods
MINOR: activity/memprofile: also monitor strdup() activity
MINOR: activity/memprofile: monitor non-portable calls as well
MINOR: activity: interrupt the show profile dump more often
MINOR: tools: resolve main() only once in resolve_sym_name()
MINOR: tools: add a new function "resolve_dso_name" to find a symbol's DSO
MINOR: activity/memprofile: use resolve_dso_name() for the DSO summary
REGTESTS: relax strerror matching to avoid a failure on libmusl
REGTESTS: don't rely on the base64 utility when openssl base64 is already
used
---
