Hi,
HAProxy 2.3.15 was released on 2021/11/04. It added 88 new commits
after version 2.3.14.
It's been almost two months without a 2.3 release now, and reports on it
are already fading away given that most users are on LTS versions. This
version collects the fixes that already went into 2.4.5 to 2.4.8, thus
for simplicity I'm copying some parts from the previous announces, so if
you have a deja-vu feeling, that's normal:
- if an HTTP/1 message was blocked for analysis waiting for some more
room, sometimes it could remain stuck indefinitely, leaving a few
never-expiring entries in "show sess".
- A very old bug was fixed in the Lua part. The wrong function was used
to start Lua tasks leading to a process freeze if the call was
performed when the time was wrapping, one millisecond every 49.7
days. On this exact millisecond, a lua task was able to be queued
with no expiration date, preventing all subsequent timers from being
seen as expired.
- A rare possibility to divide by zero in the leastconn balance
algorithm because of a thread-unsafe use of a shared variable was
fixed.
- Some bugs were fixed on the filters management to properly handle
client aborts and to be sure to always release allocated filters when
a stream is released.
- The LDAP health-check was fixed to make it compatible with Active
Directory servers. The response parsing was improved to also support
servers using multi-bytes length-encoding. Active Directory servers
seems to systematically encode messages or elements length on 4 bytes
while others are using 1-byte length-encoding if possible. Now, 1, 2
and 4 bytes length-encoding are now supported. It should be good
enough to enable LDAP health-check on Active Directory servers.
- Occasional crashes in malloc_trim() on recent glibc when running with
jemalloc were worked around.
- The build system was improved in many ways. Several -Wundef warnings
were fixed.
- HTTP "TE" header is now sanitized when a request is sent to a server.
Only "trailers" token is sent. It is mandatory because HAProxy only
understand chunked encoding. Other transfer encoding are not
supported.
- A bug on health-check was fixed when a sample fetch depending on the
execution context was used in a tcpcheck rulesets defined in a
defaults section.
- tcp-request and tcp-response content rules evaluation is now
interrupted if a read error or the end of input is detected on the
corresponding channel. This change fixes a known bug in HAProxy 2.3
and prior. However, it does not seem to affect 2.4.
- errors detected in the expression parser used with the "set-var"
action weren't detailed enough and sometimes misleading, often
leaving doubts about what to fix. This was fixed.
- resolvers: there were a large number of structural issues in the
code, and quite frankly we're not proud of the solutions but it's
impossible to do something more elegant in the current state without
a major rewrite. So what matters here is that all race conditions are
addressed and that the code works reliably. While the 2.5 fixes add a
lookup tree to perform significant CPU savings on SRV records, that
code was not backported because it adds further changes that do not
seem necessary in the current situation. We got the confirmation from
one of the reporters that the issue is now fixed.
- an interesting bug in the ring API caused boundary checks for the
wrapping at the end of the buffer to be shifted by one both in the
producer and the consumer, thus they both cancel each other and are
not observable... until the byte after the buffer is not mapped or
belongs to another area. One crash was met on boot (since startup
messages are duplicated into a ring for later retrieval), and it is
possible that those sending logs over TCP might have faced it as
well, otherwise it's extremely unlikely to be observed outside of
these use cases.
- using the tarpit could lead to head-of-line blocking of an H2
connection as the pending data were not drained. And in other
protocols, the presence of these pending data could cause a wakeup
loop between the mux and the stream, which usually ended in the
process being detected as faulty and being killed by the safety
checks.
- a similar wakeup loop could also happen when waiting for more data
(e.g. option http-buffer-request) with lots of data already present
in the receive buffer while the lower layer could only deliver a full
block at once, that couldn't fit.
- the h2spec tests in the CI were regularly failing on a few tests
expecting HTTP/2 GOAWAY frames that were sent (even seen in strace).
The problem was that we didn't perform a graceful shutdown and that
this copes badly with bidirectional communications as unread pending
data cause the connection to be reset and the frame to be lost. This
was addressed by performing a clean shutdown. It's unlikely that
anyone ever noticed this given that this essentially happens when
communication errors are reported (i.e. when the client has little
reason to complain).
- some users complained that TLS handshakes were renewed too often in
some cases. Emeric found that with the migration to the muxes in
1.9-2.0 we've lost the clean shutdown on end of connection that's
also used to commit the TLS session cache entry. For HTTP/2 this was
addressed as a side effect of the fix above, and for HTTP/1, a fix
was produced to also perform a clean shutdown on keep-alive
connections (it used to work fine only for close ones).
- the validity checks for sample fetch functions were only applied to
the frontend capability of a proxy. This means that using a small
set of sample fetch functions (like "be_name()") in proxies that are
both a frontend and a backend ("listen" or "defaults") would lead to
a config error while it is technically valid. This problem has always
been there and never reported.
- automatic cast of variables to other types would fail to first verify
if a cast method was known, possibly causing a crash at runtime when
calling them for the first time (e.g. using a variable of type address
as an argument to strcmp() or a boolean with secure_memcmp()).
- some streams could sometimes be frozen when filters were enabled (such
as compression) and an error was raised with data still left to be
processed.
- HTTP health check could report L7 timeout when facing a parse error,
because the response is dropped before being translated to HTX, while
the check waiting for a response didn't explicitly check for a possible
end-of-input.
- http-after-response rules must stop after an "allow" action, to match
their http-response counter-part.
- the parsing of the "Authorization" header field would fail if more
than one space was present between the scheme and the value.
And like for 2.4 we've added this update to halog:
- halog updates to report headers and query strings were backported, as
these are the type of improvements expected where halog is used (i.e.
in field).
Please find the usual URLs below :
Site index : http://www.haproxy.org/
Discourse : http://discourse.haproxy.org/
Slack channel : https://slack.haproxy.org/
Issue tracker : https://github.com/haproxy/haproxy/issues
Wiki : https://github.com/haproxy/wiki/wiki
Sources : http://www.haproxy.org/download/2.3/src/
Git repository : http://git.haproxy.org/git/haproxy-2.3.git/
Git Web browsing : http://git.haproxy.org/?p=haproxy-2.3.git
Changelog : http://www.haproxy.org/download/2.3/src/CHANGELOG
Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/
Willy
---
Complete changelog :
Amaury Denoyelle (3):
BUILD: ist: prevent gcc11 maybe-uninitialized warning on istalloc
BUG/MINOR: server: allow 'enable health' only if check configured
BUILD: fix compilation on NetBSD
Christopher Faulet (36):
MINOR: lua: Add a flag on lua context to know the yield capability at run
time
BUG/MINOR: lua: Yield in channel functions only if lua context can yield
BUG/MINOR: lua: Don't yield in channel.append() and channel.set()
BUG/MINOR: stream: Don't release a stream if FLT_END is still registered
BUG/MEDIUM: http-ana: Reset channels analysers when returning an error
BUG/MINOR: filters: Always set FLT_END analyser when CF_FLT_ANALYZE flag
is set
BUG/MINOR: filters: Set right FLT_END analyser depending on channel
BUG/MEDIUM: stream-int: Don't block SI on a channel policy if EOI is
reached
Revert "REGTESTS: mark http_abortonclose as broken"
BUG/MINOR: tcpcheck: Improve LDAP response parsing to fix LDAP check
BUG/MINOR: h1-htx: Fix a typo when request parser is reset
BUG/MEDIUM: mux-h1: Adjust conditions to ask more space in the channel
buffer
BUG/MEDIUM: stream-int: Notify stream that the mux wants more room to
xfer data
BUG/MEDIUM: stream: Stop waiting for more data if SI is blocked on
RXBLK_ROOM
BUG/MINOR: mux-h1/mux-fcgi: Sanitize TE header to only send "trailers"
MINOR: arg: Be able to forbid unresolved args when building an argument
list
BUG/MINOR: tcpcheck: Don't use arg list for default proxies during parsing
BUG/MINOR: tcp-rules: Stop content rules eval on read error and
end-of-input
BUG/MEDIUM: filters: Fix a typo when a filter is attached blocking the
release
BUG/MEDIUM: http-ana: Clear request analyzers when applying redirect rule
MINOR: htx: Add an HTX flag to know when a message is fragmented
MINOR: htx: Add a function to know if the free space wraps
BUG/MEDIUM: stream-int: Defrag HTX message in si_cs_recv() if necessary
BUG/MEDIUM: mux_h2: Handle others remaining read0 cases on partial frames
BUG/MINOR: http-ana: Don't eval front after-response rules if stopped on
back
BUG/MEDIUM: stream: Keep FLT_END analyzers if a stream detects a channel
error
BUG/MEDIUM: tcpcheck: Properly catch early HTTP parsing errors
BUG/MINOR: mux-h1: Save shutdown mode if the shutdown is delayed
BUG/MEDIUM: mux-h1: Perform a connection shutdown when the h1c is released
BUG/MEDIUM: resolvers: Don't recursively perform requester unlink
BUG/MEDIUM: resolvers: Track api calls with a counter to free resolutions
BUG/MEDIUM: http-ana: Drain request data waiting the tarpit timeout
expiration
BUG/MEDIUM: stream-int: Block reads if channel cannot receive more data
BUG/MEDIUM: sample: Cumulate frontend and backend sample validity flags
DOC: config: Fix alphabetical order of fc_* samples
MINOR: stream: Improve dump of bogus streams
Dragan Dosen (1):
BUG/MINOR: http-ana: increment internal_errors counter on response error
Emeric Brun (1):
DOC: peers: fix doc "enable" statement on "peers" sections
John Roesler (1):
DOC/peers: some grammar fixes for peers 2.1 spec
Olivier Houchard (1):
MINOR: initcall: Rename __GLOBL and __GLOBL1.
Remi Tricot-Le Breton (1):
BUG/MINOR: http: Authorization value can have multiple spaces after the
scheme
Tim Duesterhus (6):
MINOR: halog: Add -qry parameter allowing to preserve the query string in
-uX
DOC: halog: Move the `-qry` parameter into the correct section in help
text
MINOR: halog: Rename -qry to -query
CLEANUP: halog: Use consistent indentation in help()
BUG/MINOR: halog: Add missing newlines in die() messages
MINOR: halog: Add support for extracting captures using -hdr
William Lallemand (2):
BUG/MINOR: systemd: ExecStartPre must use -Ws
DOC: management: certificate files must be sanitized before injection
Willy Tarreau (36):
BUG/MINOR: compat: make sure __WORDSIZE is always defined
CLEANUP: pools: factor all malloc_trim() calls into trim_all_pools()
MINOR: pools: automatically disable malloc_trim() with external allocators
MINOR: pools: use mallinfo2() when available instead of mallinfo()
BUG/MINOR: cli/payload: do not search for args inside payload
BUG/MEDIUM: lua: fix wakeup condition from sleep()
BUG/MAJOR: lua: use task_wakeup() to properly run a task once
MINOR: tasks: catch TICK_ETERNITY with BUG_ON() in __task_queue()
BUG/MEDIUM: leastconn: fix rare possibility of divide by zero
CLEANUP: sample: rename sample_conv_var2smp() to *_sint
CLEANUP: sample: uninline sample_conv_var2smp_str()
MINOR: sample: provide a generic var-to-sample conversion function
BUG/MEDIUM: sample: properly verify that variables cast to sample
MINOR: resolvers: fix the resolv_str_to_dn_label() API about trailing zero
BUG/MEDIUM: resolver: make sure to always use the correct hostname length
BUG/MINOR: resolvers: do not reject host names of length 255 in SRV
records
MINOR: resolvers: fix the resolv_dn_label_to_str() API about trailing zero
BUG/MEDIUM: resolvers: fix truncated TLD consecutive to the API fix
BUG/MEDIUM: resolvers: use correct storage for the target address
MINOR: resolvers: merge address and target into a union "data"
BUG/MAJOR: resolvers: add other missing references during resolution
removal
BUILD: resolvers: avoid a possible warning on null-deref
BUG/MEDIUM: resolvers: always check a valid item in query_list
BUG/MAJOR: buf: fix varint API post- vs pre- increment
BUG/MINOR: mux-h2: do not prevent from sending a final GOAWAY frame
CLEANUP: resolvers: do not export resolv_purge_resolution_answer_records()
CLEANUP: always initialize the answer_list
CLEANUP: resolvers: simplify resolv_link_resolution() regarding requesters
CLEANUP: resolvers: replace all LIST_DELETE with LIST_DEL_INIT
MEDIUM: resolvers: use a kill list to preserve the list consistency
MEDIUM: resolvers: remove the last occurrences of the "safe" argument
BUG/MINOR: vars: improve accuracy of the rules used to check expression
validity
MINOR: sample: add missing ARGC_ entries
BUG/MINOR: vars: properly set the argument parsing context in the
expression
BUG/MINOR: sample: fix backend direction flags consecutive to last fix
SCRIPTS: git-show-backports: re-enable file-based filtering
---
