Hi, HAProxy 2.9-dev2 was released on 2023/07/21. It added 91 new commits after version 2.9-dev1. This remains pretty calm for 3 weeks, because everyone seems to be busy making progress on complex stuff thanks to 2.8 appearing to be pretty good for now.
There are now a number of low-importance fixes in queue (deadlock on listeners, a few issues with QUIC, etc). To give an idea of the extent of fixes for now, one of them removes an unneeded '\n' in some error essages... We've seen more dramatic changes :-) Now regarding the new stuff: - small rework of the sample expression parser to unify the converter and sample-fetch parser in order to remove some code duplication. As usual, no regression expected but please report any you'd notice. A second more sensitive change concerns the introduction of a sample type "same" that is not exposed but only used in declarations. This solves a problem with a few converters such as "debug()" that return the same type as the input but could only be declared with output type "any", which prevents parsing error from being detected if an incompatible converter is placed last. At this point only the "debug" converter was changed. Again, no visible change is expected unless the config is already broken. Finally, to conclude the sample conversion, sample fetch functions that used to return both IPv4 and IPv6 such as "src" etc now return a type "addr". The rationale behind this is that we used to cheat by declaring them as returning only one type and letting the converters cast on the fly. The "addr" type was only used to resolve the chain of casts in the expression. Now these ones can just return a type addr and the converter behind automatically handles it. Same, theoretically there should be no visible effect. - A new pair of SSL sample fetch functions were added: ssl_fc_curve and ssl_bc_curve. These ones are only available with OpenSSL 3.0 and above, and they are used to return the name of the curve used during the key exchange. - In Lua, it's now possible to read "proc" variables from the core API, it was found to be convenient for various things such as global thresholds or dynamic modes that can be adjusted on the fly using the CLI. Also the Lua-based mailers now handle the "timeout mail" directive from the mailers section. - several QUIC fixes, updates and traces, as usual. - SSL: after the discussion about whether or not to integrate the QUIC compatibility layer to allow QUIC to be tested on top of OpenSSL, it appeared that both here on this list and on the QUIC workgroup, the general sentiment was rather against having this for basically 3 reasons that align with my initial concerns: - would make users imagine that what they're testing is really QUIC while it can be of lower quality - would make the stubborn openssl team win the QUIC battle, confirming they were right to piss off the whole world - better aim the resources on supporting alternate libs I generally agree with the first two. I don't agree with the 3rd one because that's already what we've been doing for quite a while and are still continuing, but despite our past and ongoing efforts and commitments on this, it only very slowly improves the situation and doesn't solely depend on us, but also library implementers, their available resources, the ability for distros to adopt new components, etc. We haven't reduce our efforts on this, we do sincerely hope that LTS distros shipping next year will support wolfSSL out of the box, and maybe AWS-LC, who knows so we expect to have more choice. But this doesn't solve the issue users are facing right now. And I've received quite a few comments over the last year from people saying "do you imagine we can build quictls in field?". Of course I know it's not possible at plenty of places. I also understand that such people are not actively looking for QUIC deployments right now, but rather to validate the compability with their infrastructure and start to run tests. Some are worried that fragmentation may happen over VPN (we can say "don't worry" but they'll believe it when seeing it). Others about their firewall's timeouts impacts on bidirectional UDP streams, etc. Finally, the QUIC maintainers themselves would welcome bug reports caused by whatever bug may be left. So this tends to indicate that there is a demand for having something very close to QUIC with much less deployment trouble, and without fooling the users into thinking that it's a real QUIC. For now we've thought that enabling this mechanism only at build time via USE_QUIC_OPENSSL_COMPAT=1 and in the config as well with a directive "limited-quic" in the global section could be reasonable fit. Maybe we could rename the option to make it sound even more limited (we thought about "reduced", "degraded", "unsafe", "unsupported" etc, it's hard to find a name). The idea here is to convey the idea that if you don't like it it's rather because of the limited support than the protocol itself, while allowing users to start to discover it. I don't know how this will evolve for 2.9, maybe it will be refined, reworked, reverted or improved, I don't know. Also, the NGINX team said they're currently working on trying to get 0-RTT to work, so the long-term question becomes even more justified. Last point, in case they'd be any doubt about the intent to find an alternate SSL library, let me remind everyone that OpenSSL 3.x's disastrous performance is a much bigger problem than its lack of QUIC support, and that it's not just a QUIC patchset that will fix it, so the work on an alternative is not going to fade away. And that's roughly all for this version. I'm aware of some progress being made on various topics at the moment, hopefully we'll get some of them in dev3 or dev4: - a number of log-format tags are receiving an equivalent sample-fetch so that it will finally be possible to exploit such metrics differently if needed. Many are still missing, that's a long task due to some specific formats. - the mux-to-mux forwarding that takes care of congestion window to avoid filling buffers with blocked data is progressing. - the mechanism to automatically bind threads and create thread-groups based on CPU topology is making progress as well, hopefully by next -dev we'll be able to see optimal binding on multi-socket machines, segmented caches and hybrid CPUs. - the patch series to reduce the locking contention on stick-tables should be merged soon - some work was finally started to reduce the locking cost in the shctx blocks used by the cache. I don't have the rest in mind at the moment. Please find the usual URLs below : Site index : https://www.haproxy.org/ Documentation : https://docs.haproxy.org/ Wiki : https://github.com/haproxy/wiki/wiki Discourse : https://discourse.haproxy.org/ Slack channel : https://slack.haproxy.org/ Issue tracker : https://github.com/haproxy/haproxy/issues Sources : https://www.haproxy.org/download/2.9/src/ Git repository : https://git.haproxy.org/git/haproxy.git/ Git Web browsing : https://git.haproxy.org/?p=haproxy.git Changelog : https://www.haproxy.org/download/2.9/src/CHANGELOG Dataplane API : https://github.com/haproxytech/dataplaneapi/releases/latest Pending bugs : https://www.haproxy.org/l/pending-bugs Reviewed bugs : https://www.haproxy.org/l/reviewed-bugs Code reports : https://www.haproxy.org/l/code-reports Latest builds : https://www.haproxy.org/l/dev-packages Willy --- Complete changelog : Aurelien DARRAGON (27): BUG/MINOR: tcp_sample: bc_{dst,src} return IP not INT MEDIUM: acl/sample: unify sample conv parsing in a single function MINOR: sample: introduce c_pseudo() conv function MEDIUM: sample: add missing ADDR=>? compatibility matrix entries MINOR: sample: fix ipmask sample definition MEDIUM: tree-wide: fetches that may return IPV4+IPV6 now return ADDR MEDIUM: sample: introduce 'same' output type BUG/MEDIUM: sink: invalid server list in sink_new_from_logsrv() BUG/MINOR: http_ext: unhandled ERR_ABORT in proxy_http_parse_7239() BUG/MINOR: sink: missing sft free in sink_deinit() BUG/MINOR: ring: size warning incorrectly reported as fatal error BUG/MINOR: ring: maxlen warning reported as alert BUG/MINOR: log: LF upsets maxlen for UDP targets MINOR: sink/api: pass explicit maxlen parameter to sink_write() BUG/MEDIUM: log: improper use of logsrv->maxlen for buffer targets BUG/MINOR: log: fix missing name error message in cfg_parse_log_forward() BUG/MINOR: log: fix multiple error paths in cfg_parse_log_forward() BUG/MINOR: log: free errmsg on error in cfg_parse_log_forward() BUG/MINOR: sink: invalid sft free in sink_deinit() BUG/MINOR: sink: fix errors handling in cfg_post_parse_ring() BUG/MINOR: server: set rid default value in new_server() MINOR: hlua_fcn/mailers: handle timeout mail from mailers section BUG/MINOR: sink/log: properly deinit srv in sink_new_from_logsrv() EXAMPLES: maintain haproxy 2.8 retrocompatibility for lua mailers script BUG/MINOR: hlua_fcn/queue: use atomic load to fetch queue size BUG/MINOR: hlua: hlua_yieldk ctx argument should support pointers BUG/MEDIUM: hlua_fcn/queue: bad pop_wait sequencing Christopher Faulet (6): DOC: config: Fix fc_src description to state the source address is returned BUG/MINOR: sample: Fix wrong overflow detection in add/sub conveters BUG/MINOR: http: Return the right reason for 302 BUG/MINOR: h1-htx: Return the right reason for 302 FCGI responses BUG/MINOR: server: Don't warn on server resolution failure with init-addr none BUG/MEDIUM: listener: Acquire proxy's lock in relax_listener() if necessary Daan van Gorkum (1): MINOR: lua: Allow reading "proc." scoped vars from LUA core. Emeric Brun (6): BUG/MEDIUM: quic: token IV was not computed using a strong secret BUG/MINOR: quic: retry token remove one useless intermediate expand BUG/MEDIUM: quic: missing check of dcid for init pkt including a token BUG/MEDIUM: quic: timestamp shared in token was using internal time clock CLEANUP: quic: remove useless parameter 'key' from quic_packet_encrypt BUILD: quic: fix warning during compilation using gcc-6.5 Frédéric Lécaille (33): BUG/MINOR: quic: Possible leak when allocating an encryption level BUG/MINOR: quic: Missing QUIC connection path member initialization BUILD: quic: Compilation fixes for some gcc warnings with -O1 BUG/MINOR: quic: Possible crash in "show quic" dumping packet number spaces BUG/MINOR: quic: Unckecked encryption levels availability MINOR: quic: Stop storing the TX encoded transport parameters MINOR: quic: Dynamic allocation for negotiated Initial TLS cipher context. MINOR: quic: Release asap the negotiated Initial TLS context. MINOR: quic: Add traces to qc_may_build_pkt() MEDIUM: quic: Packet building rework. CLEANUP: quic: Remove a useless TLS related variable from quic_conn_io_cb(). MEDIUM: quic: Handshake I/O handler rework. MINOR: quic: Add traces for qc_frm_free() MINOR: quic: add trace about pktns packet/frames releasing BUG/MINOR: quic: Missing parentheses around PTO probe variable. MINOR: quic: Ping from Initial pktns before reaching anti-amplification limit MINOR: quic: QUIC openssl wrapper implementation MINOR: quic: Include QUIC opensssl wrapper header from TLS stacks compatibility header MINOR: quic: Do not enable O-RTT with USE_QUIC_OPENSSL_COMPAT MINOR: quic: Set the QUIC connection as extra data before calling SSL_set_quic_method() MINOR: quic: Do not enable 0RTT with SSL_set_quic_early_data_enabled() MINOR: quic: Add a compilation option for the QUIC OpenSSL wrapper MINOR: quic: Export some KDF functions (QUIC-TLS) MINOR: quic: Make ->set_encryption_secrets() be callable two times MINOR: quic: Initialize TLS contexts for QUIC openssl wrapper MINOR: quic: Call the keylog callback for QUIC openssl wrapper from SSL_CTX_keylog() MINOR: quic: Add a quic_openssl_compat struct to quic_conn struct MINOR: quic: Useless call to SSL_CTX_set_quic_method() MINOR: quic: SSL context initialization with QUIC OpenSSL wrapper. MINOR: quic: Missing encoded transport parameters for QUIC OpenSSL wrapper MINOR: quic: Add "limited-quic" new tuning setting DOC: quic: Add "limited-quic" new tuning setting DOC: install: Document how to build a limited support for QUIC Ilya Shipitsin (2): CI: add naming convention documentation CI: explicitely highlight VTest result section if there's something Marcos de Oliveira (2): BUG/MINOR: server-state: Ignore empty files BUG/MINOR: server-state: Avoid warning on 'file not found' Mariam John (1): MEDIUM: ssl: new sample fetch method to get curve name Patrick Hemmer (1): MINOR: peers: add peers keyword registration Remi Tricot-Le Breton (3): DOC: ssl: Fix typo in 'ocsp-update' option DOC: ssl: Add ocsp-update troubleshooting clues and emphasize on crt-list only aspect BUG/MINOR: cache: A 'max-age=0' cache-control directive can be overriden by a s-maxage Thierry Fournier (2): BUG/MINOR: config: Remove final '\n' in error messages BUG/MINOR: config: Lenient port configuration parsing Willy Tarreau (6): MINOR: cpuset: add cpu_map_configured() to know if a cpu-map was found BUG/MINOR: config: do not detect NUMA topology when cpu-map is configured BUG/MINOR: cpuset: remove the bogus "proc" from the cpu_map struct BUG/MINOR: init: set process' affinity even in foreground CLEANUP: cpuset: remove the unused proc_t1 field in cpu_map CLEANUP: config: make parse_cpu_set() return documented values firexinghe (1): BUG/MINOR: hlua: add check for lua_newstate ---