[ANNOUNCE] haproxy-2.0.23

[Christopher Faulet]

Hi,

HAProxy 2.0.23 was released on 2021/07/16. It added 97 new commits
after version 2.0.22.

All commits were already mentioned in announcements of the last 2.3 and 2.2
releases. So I will be brief. Here is the list of main bugs fixed by this
release:

  * An issue in the CONTINUATION frames parsing in h2 leading to spurious
wakeups.

  * A bug in the shutdowns detection for h2 connections.

  * A possible deadlock if "set maxconn server" command was used when there
was a pending connection ready to be dequeued.

  * A bug in the HTX defragmentation leading to crash. The bug might be
encountered in the HTTP compression filter or in HTTP header
replacement.

  * An old bug preventing the dequeuing for servers with a very low maxconn
because the load balancing was not skipped when a new connection was
picked from the proxy's or server's queue.

  * A bug in the sock part leading to high CPU usage because some early
connection failures might be missed.

  * A thread-safety issue with the SHCTX code when compiled with
USE_PRIVATE_CACHE mode. It was not using any locks.

  * An issue with the abortonclose option. It was not working since a while.

  * A bug in the HTTP compression leading to truncated or corrupted
responses.

  * "url_ip"/"url_port" sample fetches not properly handling url parsing
errors.

  * A bug in the cpu-map notation when both processes and threads were
specified, most specifically P-Q/1 or 1/P-Q notation.

  * Some issues affecting the peers synchronization.

  * H1 idle connections on server side receiving data not closed. Because of
this bug, it was possible to send pending 408-Request-time-out responses
to clients.

  * Wrong number of retries reported in logs if no connection was
attempted. Since the beginning, when the session was aborted before any
connection attempt to any server, the backend retries value was
reported, instead of 0.

  * A bug with the method sample fetch when an exotic method is found.

Note that resolvers performance issues fixed in upper versions were not
fixed in this one because the code is too different. This part is a
perpetual source of bugs. It was too risky to backport the fixes. If you
experience any issue with the resolvers, please consider to upgrade to a
newer version.

Thanks everyone for your help and your contributions!

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Wiki : https://github.com/haproxy/wiki/wiki
   Sources  : http://www.haproxy.org/download/2.0/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.0.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.0.git
   Changelog: http://www.haproxy.org/download/2.0/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/


---
Complete changelog :
Amaury Denoyelle (6):
  BUG/MINOR: server: free srv.lb_nodes in free_server
  BUG/MEDIUM: config: fix cpu-map notation with both process and threads
  BUG/MINOR: http_fetch: fix possible uninit sockaddr in fetch_url_ip/port
  BUG/MAJOR: server: prevent deadlock when using 'set maxconn server'
  BUG/MINOR: stick-table: insert srv in used_name tree even with fixed id
  BUG/MAJOR: server: fix deadlock when changing maxconn via agent-check

Christopher Faulet (34):
  DOC: Explicitly state only IPv4 are supported by forwardfor/originalto 
options
  BUG/MEDIUM: threads: Ignore current thread to end its harmless period
  BUG/MINOR: http-fetch: Make method smp safe if headers were already 
forwarded
  BUG/MINOR: http_htx: Remove BUG_ON() from http_get_stline() function
  BUG/MINOR: logs: Report the true number of retries if there was no 
connection
  BUG/MINOR: mux-h1: Release idle server H1 connection if data are received
  BUG/MAJOR: mux-h2: Properly detect too large frames when decoding headers
  BUG/MEDIUM: mux-h2: Fix dfl calculation when merging CONTINUATION frames
  BUG/MEDIUM: mux-h2: Properly handle shutdowns when received with data
  BUG/MINOR: htx: Preserve HTX flags when draining data from an HTX message
  BUG/MINOR: applet: Notify the other side if data were consumed by an 
applet
  BUG/MINOR: hlua: Don't rely on top of the stack when using Lua buffers
  BUG/MINOR: stream: Decrement server current session counter on L7 retry
  BUG/MINOR: stream: Reset stream final state and si error type on L7 retry
  MINOR: channel: Rely on HTX version if appropriate in channel_may_recv()
  BUG/MINOR: stream-int: Don't block reads in si_update_rx() if chn may 
receive
  MEDIUM: mux-h1: Don't block reads when waiting for the other side
  REGTESTS: Add script to test abortonclose option
  BUG/MEDIUM: filters: Exec pre/post analysers 

[ANNOUNCE] haproxy-2.2.15

[Christopher Faulet]

Hi,

HAProxy 2.2.15 was released on 2021/07/16. It added 90 new commits
after version 2.2.14.

This release is very similar to the 2.3.11/2.3.12. To sum up, most
noticeable bugs fixed in this release are:

  * A possible deadlock if "set maxconn server" command was used when there
was a pending connection ready to be dequeued.

  * A possible infinite loop in process_stream() when a connection error was
reported while the stream was waiting for a retry.

  * A possible race between free() and pool_alloc() in the pools lockless
variant.

  * A bug in the HTX defragmentation leading to crash. The bug might be
encountered in the HTTP compression filter or in HTTP header
replacement.

  * An old bug preventing the dequeuing for servers with a very low maxconn
because the load balancing was not skipped when a new connection was
picked from the proxy's or server's queue.

  * A bug in the sock part leading to high CPU usage because some early
connection failures might be missed.

  * A thread-safety issue with the SHCTX code when compiled with
USE_PRIVATE_CACHE mode. It was not using any locks.

  * Most of resolvers performance issues and several other bugs in this area.

  * An issue with the abortonclose option. It was not working since a while.

  * A bug in the HTTP compression leading to truncated or corrupted
responses.

  * A bug with synchronous connect in tcpcheck when several connections come
one after the other.

  * "url_ip"/"url_port" sample fetches not properly handling url parsing
errors.

In addition, the http-ignore-probes is now respected for H2
connections. When this option is set, no errors are reported anymore when
connections are aborted during preface. And the FCGI multiplexer was
slightly improved to send a relative path instead of a normalized URI to an
application and to expose SERVER_SOFTWARE parameter by default. Finally, as
a consequence of the bug fixed in the pools, the code was simplified. The
lockless implementation is used everywhere, resulting in the removal of the
very old locked implementation that was kept for non-capable
architectures. As a result, threads will now be faster on less common
architectures (e.g. i686, MIPS, PPC64, ...). The rest is less visible but
contains, as usual, cleanups, small fixes here and there, improvements...

It is strongly advised to update to this version. Thanks everyone for your
help and your contributions!

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Wiki : https://github.com/haproxy/wiki/wiki
   Sources  : http://www.haproxy.org/download/2.2/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.2.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.2.git
   Changelog: http://www.haproxy.org/download/2.2/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/


---
Complete changelog :
Alex (1):
  DOC: use the req.ssl_sni in examples

Alexandar Lazic (1):
  DOC/MINOR: move uuid in the configuration to the right alphabetical order

Amaury Denoyelle (5):
  BUG/MINOR: http_fetch: fix possible uninit sockaddr in fetch_url_ip/port
  BUG/MAJOR: server: prevent deadlock when using 'set maxconn server'
  BUG/MINOR: stick-table: insert srv in used_name tree even with fixed id
  BUG/MAJOR: server: fix deadlock when changing maxconn via agent-check
  REGTESTS: fix maxconn update with agent-check

Christopher Faulet (35):
  BUG/MINOR: hlua: Don't rely on top of the stack when using Lua buffers
  BUG/MINOR: stream: Decrement server current session counter on L7 retry
  BUG/MINOR: stream: Reset stream final state and si error type on L7 retry
  BUG/MINOR: checks: Handle synchronous connect when a tcpcheck is started
  BUG/MINOR: checks: Reschedule check on observe mode only if fastinter is 
set
  MINOR: channel: Rely on HTX version if appropriate in channel_may_recv()
  BUG/MINOR: stream-int: Don't block reads in si_update_rx() if chn may 
receive
  MINOR: conn-stream: Force mux to wait for read events if abortonclose is 
set
  MEDIUM: mux-h1: Don't block reads when waiting for the other side
  BUG/MEDIUM: mux-h1: Properly report client close if abortonclose option 
is set
  REGTESTS: Add script to test abortonclose option
  BUG/MEDIUM: filters: Exec pre/post analysers only one time per filter
  BUG/MINOR: http-comp: Preserve HTTP_MSGF_COMPRESSIONG flag on the response
  BUG/MINOR: http-ana: Handle L7 retries on refused early data before K/A 
aborts
  BUG/MAJOR: stream-int: Release SI endpoint on server side ASAP on retry
  BUG/MEDIUM: compression: Add a flag to know the filter is still 
processing data
  BUG/MAJOR: htx: Fix htx_defrag() when an 

Re: set mss on backend site on version 1.7.9

[Stefan Fuhrmann]

Hello Lukas,


okay, thanks!!


Stefan


Am 13.07.21 um 20:49 schrieb Lukas Tribus:

Hello Stefan,

On Tue, 13 Jul 2021 at 14:10, Stefan Fuhrmann
 wrote:

Hello all,


First, we can not change to newer version so fast within the project.

We are having on old installation of haproxy (1.7.9) and we have the
need to configure tcp- mss- value on backend site.



Is that possible to change the mss- value on backend site? How?

No.

You can set the MSS on the frontend socket, but not on the backend socket.

You need to work with your OS/kernel configuration.


Lukas

Re: Replying to spam [was: Some Spam Mail]

[Lukas Tribus]

On Thu, 15 Jul 2021 at 11:27, Илья Шипицин  wrote:
>
> I really wonder what they will suggest.
>
> I'm not a spam source, since we do not have "opt in" policy, anybody can send 
> mail. so they do.
> please address the issue properly, either change list policy or be calm with 
> my experiments.

It's about common sense, not list policy. Please do your SPAM
responding experiments without the list in CC.


Thank you,

Lukas

Re: [External] Re: [PATCH] JA3 TLS Fingerprinting (take 2)

[Marcin Deranek]

Hi Tim,

Updated (see attachments). Other patches did not change.
Regards,

Marcin Deranek

On Thu, Jul 15, 2021 at 10:20 AM Tim Düsterhus  wrote:

> Marcin,
>
> On 7/14/21 2:01 PM, Marcin Deranek wrote:
> > Thank you for all comments I have received regarding JA3 Fingerprinting
> > patches. Here is the new set of patches which incorporated all your
> > suggestions.
>
> Sorry I gave a little outdated advice regarding the reg-tests. For any
> new tests please use:
>
>feature cmd "$HAPROXY_PROGRAM -cc 'version_atleast(2.5-dev0)'"
>
> instead of
>
>#REQUIRE_VERSION=2.5
>
> Other than that the tests LGTM from a glance. I didn't look at your C
> and I also didn't (yet) compare the tests against the documentation you
> have written.
>
> Best regards
> Tim Düsterhus
>


-- 
Marcin Deranek
Senior Site Reliability Engineer
[image: Booking.com] 
Making it easier for everyone
to experience the world.
From 87dcccaa9d3fbd4474b256bf6623323621c3144b Mon Sep 17 00:00:00 2001
From: Marcin Deranek 
Date: Tue, 13 Jul 2021 14:05:24 +0200
Subject: [PATCH 3/5] MINOR: sample: Add be2dec converter

Add be2dec converter which allows to build JA3 compatible TLS
fingerprints by converting big-endian binary data into string
separated unsigned integers eg.

http-request set-header X-SSL-JA3 %[ssl_fc_protocol_hello_id],\
%[ssl_fc_cipherlist_bin(1),be2dec(-,2)],\
%[ssl_fc_extlist_bin(1),be2dec(-,2)],\
%[ssl_fc_eclist_bin(1),be2dec(-,2)],\
%[ssl_fc_ecformats_bin,be2dec(-,1)]
---
 doc/configuration.txt  | 12 +++
 reg-tests/converter/be2dec.vtc | 56 +
 src/sample.c   | 57 ++
 3 files changed, 125 insertions(+)
 create mode 100644 reg-tests/converter/be2dec.vtc

diff --git a/doc/configuration.txt b/doc/configuration.txt
index ecbbcdd04..d39e90752 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -16064,6 +16064,18 @@ base64
   an SSL ID can be copied in a header). For base64url("URL and Filename
   Safe Alphabet" (RFC 4648)) variant see "ub64enc".
 
+be2dec(,,[])
+  Converts a binary input sample to a string containing an unsigned integer
+  number per  input bytes.  is put every 
+  binary input bytes if specified.  flag indicates whatever binary
+  input is truncated at  boundaries.  maximum value is
+  limited by the size of long long int (8 bytes).
+
+  Example:
+  bin(01020304050607),be2dec(:,2)   # 258:772:1286:7
+  bin(01020304050607),be2dec(-,2,1) # 258-772-1286
+  bin(01020304050607),be2dec(,2,1)  # 2587721286
+
 bool
   Returns a boolean TRUE if the input value of type signed integer is
   non-null, otherwise returns FALSE. Used in conjunction with and(), it can be
diff --git a/reg-tests/converter/be2dec.vtc b/reg-tests/converter/be2dec.vtc
new file mode 100644
index 0..bdb903523
--- /dev/null
+++ b/reg-tests/converter/be2dec.vtc
@@ -0,0 +1,56 @@
+varnishtest "be2dec converter Test"
+
+feature cmd "$HAPROXY_PROGRAM -cc 'version_atleast(2.5-dev0)'"
+feature ignore_unknown_macro
+
+server s1 {
+	rxreq
+	txresp
+} -repeat 3 -start
+
+haproxy h1 -conf {
+defaults
+	mode http
+	timeout connect 1s
+	timeout client  1s
+	timeout server  1s
+
+frontend fe
+	bind "fd@${fe}"
+
+	 requests
+	http-request  set-var(txn.input) req.hdr(input)
+
+	http-response set-header be2dec-1   "%[var(txn.input),be2dec(:,1)]"
+	http-response set-header be2dec-2   "%[var(txn.input),be2dec(-,3)]"
+	http-response set-header be2dec-3   "%[var(txn.input),be2dec(::,3,1)]"
+
+	default_backend be
+
+backend be
+	server s1 ${s1_addr}:${s1_port}
+} -start
+
+client c1 -connect ${h1_fe_sock} {
+	txreq -url "/" \
+	  -hdr "input:"
+	rxresp
+	expect resp.status == 200
+	expect resp.http.be2dec-1 == ""
+	expect resp.http.be2dec-2 == ""
+	expect resp.http.be2dec-3 == ""
+	txreq -url "/" \
+	  -hdr "input: 0123456789"
+	rxresp
+	expect resp.status == 200
+	expect resp.http.be2dec-1 == "48:49:50:51:52:53:54:55:56:57"
+	expect resp.http.be2dec-2 == "3158322-3355701-3553080-57"
+	expect resp.http.be2dec-3 == "3158322::3355701::3553080"
+	txreq -url "/" \
+	  -hdr "input: abcdefghijklmnopqrstuvwxyz"
+	rxresp
+	expect resp.status == 200
+	expect resp.http.be2dec-1 == "97:98:99:100:101:102:103:104:105:106:107:108:109:110:111:112:113:114:115:116:117:118:119:120:121:122"
+	expect resp.http.be2dec-2 == "6382179-6579558-6776937-6974316-7171695-7369074-7566453-7763832-31098"
+	expect resp.http.be2dec-3 == "6382179::6579558::6776937::6974316::7171695::7369074::7566453::7763832"
+} -run
diff --git a/src/sample.c b/src/sample.c
index d02034cf0..5b7ad8b34 100644
--- a/src/sample.c
+++ b/src/sample.c
@@ -2057,6 +2057,62 @@ static int sample_conv_crypto_hmac(const struct arg *args, struct sample *smp, v
 
 #endif /* USE_OPENSSL */
 
+static int sample_conv_be2dec_check(struct arg *args, struct sample_conv *conv,
+const char *file, int line, char **err)
+{
+	if (arg

Re: Replying to spam [was: Some Spam Mail]

[Илья Шипицин]

I really wonder what they will suggest.

I'm not a spam source, since we do not have "opt in" policy, anybody can
send mail. so they do.
please address the issue properly, either change list policy or be calm
with my experiments.

чт, 15 июл. 2021 г. в 12:51, Tim Düsterhus :

> Ilya,
>
> On 7/14/21 5:20 PM, Илья Шипицин wrote:
> > Yes, go ahead
>
> Would you please stop replying to spam, especially with both the sender
> and the list in Cc? It just causes more spam down the road.
>
> Best regards
> Tim Düsterhus
>

Re: [PATCH] JA3 TLS Fingerprinting (take 2)

[Tim Düsterhus]

Marcin,

On 7/14/21 2:01 PM, Marcin Deranek wrote:

Thank you for all comments I have received regarding JA3 Fingerprinting
patches. Here is the new set of patches which incorporated all your
suggestions.


Sorry I gave a little outdated advice regarding the reg-tests. For any 
new tests please use:


  feature cmd "$HAPROXY_PROGRAM -cc 'version_atleast(2.5-dev0)'"

instead of

  #REQUIRE_VERSION=2.5

Other than that the tests LGTM from a glance. I didn't look at your C 
and I also didn't (yet) compare the tests against the documentation you 
have written.


Best regards
Tim Düsterhus

Replying to spam [was: Some Spam Mail]

[Tim Düsterhus]

Ilya,

On 7/14/21 5:20 PM, Илья Шипицин wrote:

Yes, go ahead


Would you please stop replying to spam, especially with both the sender 
and the list in Cc? It just causes more spam down the road.


Best regards
Tim Düsterhus