Re: [PATCH] BUILD: ssl: Build with new cryptographic library AWS-LC

[Hopkins, Andrew]

I tried it and HAProxy doesn’t build with AWS-LC when quic is turned on. There 
are at least two issues:
1. AWS-LC’s TLS 1.3 cipher suite names are a little different, this is easy to 
fix and I opened https://github.com/aws/aws-lc/pull/1175
2. ChaCha Poly and AES CCM are not usable through the EVP_CIPHER API, AWS-LC 
only exposes these through the AEAD API

How important is ChaCha Poly & AES CCM to HAProxy and your users? I see three 
options:
1. AWS-LC plumbs these two algorithms through the EVP_CIPHER API. This is 
useful for HAProxy and other AWS-LC customers, but is the most work
2. HAProxy adopts AWS-LC’s (and BoringSSL’s) AEAD API
3. HAProxy turns off ChaCha Poly and AES CCM support in quic when built with 
AWS-LC


From: Илья Шипицин 
Date: Wednesday, September 6, 2023 at 5:41 AM
To: William Lallemand 
Cc: "Hopkins, Andrew" , Willy Tarreau , 
Aleksandar Lazic , "haproxy@formilux.org" 

Subject: RE: [EXTERNAL] [PATCH] BUILD: ssl: Build with new cryptographic 
library AWS-LC


CAUTION: This email originated from outside of the organization. Do not click 
links or open attachments unless you can confirm the sender and know the 
content is safe.


based on USE_OPENSSL_AWSLC quic may be enabled  ?

ср, 6 сент. 2023 г. в 14:26, William Lallemand 
mailto:wlallem...@haproxy.com>>:
On Tue, Sep 05, 2023 at 11:56:26PM +, Hopkins, Andrew wrote:
> I split up the remaining CI changes into 4 new attached patches. The
> latest changes are still passing on my fork
> https://github.com/andrewhop/haproxy/actions/runs/6090899582.
>

Thanks, I just merged them!


> I was hoping to take advantage of the better HAProxy support in
> AWS-LC's CI but I'm running into some issues in
> https://github.com/aws/aws-lc/pull/1174 I was wondering if you had any
> pointers of what to look at. I think this is CodeBuild specific issue
> since the tests pass in HAProxy's CI and when I run AWS-LC's CI
> locally. I just can't figure out what CodeBuild might be doing to mess
> with the results.
>
> Looking at the log for mcli_start_progs.vtc the two sleep programs are
> started as expected but the overall process returns the wrong exit
> code (0x0 instead of 0x82). Does anything stand out to you as weird
> looking?
>

I never used CodeBuild so I'm not aware on any timers or process
limitation but that could be something like that.

From what I understand from the trace, I think every processes received a
SIGTERM. You can see 2 "Exiting Master process..." and the first one is before
the "kill" from VTest which is suppose to send a SIGINT so it was probably sent
outside the test.

This test should finish like this:

***  h1debug|:MASTER.accept(0008)=000e from 
[127.0.0.1:41542] ALPN=
***  h1debug|:MASTER.srvcls[000e:]
 h1CLI connection normally closed
***  h1CLI closing fd 9
 h1CLI recv|#   

 h1CLI recv|357949  master  0 [failed: 0]   0d00h00m00s 
2.9-dev4-06d369-78
 h1CLI recv|# workers
 h1CLI recv|357955  worker  0   0d00h00m00s 
2.9-dev4-06d369-78
 h1CLI recv|# programs
 h1CLI recv|357953  foo 0   0d00h00m00s 
-
 h1CLI recv|357954  bar 0   0d00h00m00s 
-
 h1CLI recv|
***  h1debug|0001:MASTER.clicls[:]
***  h1debug|0001:MASTER.closed[:]
 h1CLI expect match ~ ".*foo.*
.*bar.*
"
**   h1CLI ending
**   h1Wait
**   h1Stop HAproxy pid=357949
 h1Kill(2)=0: Success
***  h1debug|[NOTICE]   (357949) : haproxy version is 2.9-dev4-06d369-78
***  h1debug|[NOTICE]   (357949) : path to executable is 
/home/wla/projects/haproxy/haproxy-community-maint/haproxy
***  h1debug|[WARNING]  (357949) : Exiting Master process...
***  h1debug|[ALERT](357949) : Current program 'foo' (357953) exited 
with code 130 (Interrupt)
***  h1debug|[ALERT](357949) : Current program 'bar' (357954) exited 
with code 130 (Interrupt)
 dT0.076
***  h1debug|[ALERT](357949) : Current worker (357955) exited with code 
130 (Interrupt)
***  h1debug|[WARNING]  (357949) : All workers exited. Exiting... (130)
 dT0.077
 h1STDOUT EOF
 dT0.171
**   h1WAIT4 pid=357949 status=0x8200 (user 0.058881 sys 0.026402)
*top   RESETTING after reg-tests/mcli/mcli_start_progs.vtc
**   h1Reset and free h1 haproxy -1
 dT0.172
**   s1Waiting for server (4/-1)
*top   TEST reg-tests/mcli/mcli_start_progs.vtc completed
*diag  0.0 /usr/bin/sleep
#top  TEST reg-tests/mcli/mcli_start_progs.vtc passed (0.173)
0 tests failed, 0 tests skipped, 1 tests passed


--
William Lallemand

Re: [PATCH] BUILD: ssl: Build with new cryptographic library AWS-LC

[Hopkins, Andrew]

Thanks for looking at this. I couldn't figure out what in CodeBuild was sending 
a different signal to the processes so I moved the HAPRoxy job to GitHub 
actions like your CI in https://github.com/aws/aws-lc/pull/1174 and it's all 
passing over here too!

On 9/6/23, 5:26 AM, "William Lallemand" mailto:wlallem...@haproxy.com>> wrote:


CAUTION: This email originated from outside of the organization. Do not click 
links or open attachments unless you can confirm the sender and know the 
content is safe.






On Tue, Sep 05, 2023 at 11:56:26PM +, Hopkins, Andrew wrote:
> I split up the remaining CI changes into 4 new attached patches. The
> latest changes are still passing on my fork
> https://github.com/andrewhop/haproxy/actions/runs/6090899582 
> .
>


Thanks, I just merged them!




> I was hoping to take advantage of the better HAProxy support in
> AWS-LC's CI but I'm running into some issues in
> https://github.com/aws/aws-lc/pull/1174 
>  I was wondering if you had any
> pointers of what to look at. I think this is CodeBuild specific issue
> since the tests pass in HAProxy's CI and when I run AWS-LC's CI
> locally. I just can't figure out what CodeBuild might be doing to mess
> with the results.
>
> Looking at the log for mcli_start_progs.vtc the two sleep programs are
> started as expected but the overall process returns the wrong exit
> code (0x0 instead of 0x82). Does anything stand out to you as weird
> looking?
>


I never used CodeBuild so I'm not aware on any timers or process
limitation but that could be something like that.


From what I understand from the trace, I think every processes received a
SIGTERM. You can see 2 "Exiting Master process..." and the first one is before
the "kill" from VTest which is suppose to send a SIGINT so it was probably sent
outside the test.


This test should finish like this:


*** h1 debug|:MASTER.accept(0008)=000e from [127.0.0.1:41542] 
ALPN=
*** h1 debug|:MASTER.srvcls[000e:]
 h1 CLI connection normally closed
*** h1 CLI closing fd 9
 h1 CLI recv|#
 h1 CLI recv|357949 master 0 [failed: 0] 0d00h00m00s 2.9-dev4-06d369-78
 h1 CLI recv|# workers
 h1 CLI recv|357955 worker 0 0d00h00m00s 2.9-dev4-06d369-78
 h1 CLI recv|# programs
 h1 CLI recv|357953 foo 0 0d00h00m00s -
 h1 CLI recv|357954 bar 0 0d00h00m00s -
 h1 CLI recv|
*** h1 debug|0001:MASTER.clicls[:]
*** h1 debug|0001:MASTER.closed[:]
 h1 CLI expect match ~ ".*foo.*
.*bar.*
"
** h1 CLI ending
** h1 Wait
** h1 Stop HAproxy pid=357949
 h1 Kill(2)=0: Success
*** h1 debug|[NOTICE] (357949) : haproxy version is 2.9-dev4-06d369-78
*** h1 debug|[NOTICE] (357949) : path to executable is 
/home/wla/projects/haproxy/haproxy-community-maint/haproxy
*** h1 debug|[WARNING] (357949) : Exiting Master process...
*** h1 debug|[ALERT] (357949) : Current program 'foo' (357953) exited with code 
130 (Interrupt)
*** h1 debug|[ALERT] (357949) : Current program 'bar' (357954) exited with code 
130 (Interrupt)
 dT 0.076
*** h1 debug|[ALERT] (357949) : Current worker (357955) exited with code 130 
(Interrupt)
*** h1 debug|[WARNING] (357949) : All workers exited. Exiting... (130)
 dT 0.077
 h1 STDOUT EOF
 dT 0.171
** h1 WAIT4 pid=357949 status=0x8200 (user 0.058881 sys 0.026402)
* top RESETTING after reg-tests/mcli/mcli_start_progs.vtc
** h1 Reset and free h1 haproxy -1
 dT 0.172
** s1 Waiting for server (4/-1)
* top TEST reg-tests/mcli/mcli_start_progs.vtc completed
* diag 0.0 /usr/bin/sleep
# top TEST reg-tests/mcli/mcli_start_progs.vtc passed (0.173)
0 tests failed, 0 tests skipped, 1 tests passed




--
William Lallemand

Re: Request for feedback: Add support for txn args as arguments in converter "bytes"

[Lokesh Jindal]

PFA the patch.

I have two questions:
1. Can you explain what this method does and why is it needed? (I used it
in my patch following the pattern in the converter "sub") -
*smp_set_owner(_arg0,
smp->px, smp->sess, smp->strm, smp->opt);*
2. In my patch, *sample_conv_bytes* returns 0 in case of an invalid arg
e.g. negative offset. Can you explain when a converter should return 0 V 1?
I am not sure how that impacts the haproxy behavior.

On Mon, Aug 28, 2023 at 9:56 AM Lokesh Jindal <15ljin...@gmail.com> wrote:

> Thanks for the response and the corrections, Willy.
>
> *We need to decide what to do when the variable does not*
>
> *exist or is empty. We can't make converters fail for now, so most
> likelyit will have to end up as value zero for offset and/or length*.
>
> Here is the implementation today - link
> .
> We set the length of the output to 0 in case of an invalid input (e.g. arg0
> value >= length of the bytes in the input to the converter).
> So, for all other invalid inputs (e.g. variable in arg[0] does not exist),
> we can do the same.
>
> We can discuss more after I share the patch.
>
> Thanks
> Lokesh
>
> On Mon, Aug 28, 2023 at 2:53 AM Willy Tarreau  wrote:
>
>> Hi Lokesh,
>>
>> On Fri, Aug 25, 2023 at 01:44:48PM -0700, Lokesh Jindal wrote:
>> > Hey folks
>> >
>> > I am writing to gather feedback on an idea before doing the
>> implementation.
>> > Per the documentation, converter "bytes" accepts integer values as
>> > arguments, but not txn args.
>> > i.e. ,bytes(2,8) will work
>> > but ,bytes(txn.start_idx,txn.length) will not work.
>> >
>> > For our use case, we need to parse some binary data (a cookie) to
>> extract
>> > some info in haproxy. However, the bytes that need to be extracted are
>> not
>> > fixed and will depend on the request. We could use simple arithmetic to
>> > determine the index/length for bytes to be extracted and store them in
>> txn
>> > args. These txn args can then be used with converter "bytes".
>> >
>> > I can see that the converter "sub" already supports txn args as
>> arguments.
>> > I have successfully validated the proposed idea with an implementation
>> > following the same pattern as "sub".
>>
>> In fact it's not "txn", it's variables in general. Most of the arithmetic
>> functions support a variable name as an alternative to a numeric value. I
>> tend to think it would indeed make sense to support both integers and
>> variables for bytes(), it could even be the same for a few other ones
>> (maybe field(), word(), ltrim(), rtrim() etc).
>>
>> > Let me know what you think. If there are no concerns, I can send a
>> patch.
>>
>> I'm all for it. We need to decide what to do when the variable does not
>> exist or is empty. We can't make converters fail for now, so most likely
>> it will have to end up as value zero for offset and/or length.
>>
>> Thanks,
>> Willy
>>
>


0001-MEDIUM-sample-Enhances-converter-bytes-to-take-varia.patch
Description: Binary data

[PATCH 0/2] CI: musl pipeline improvement

[Ilya Shipitsin]

it has been figured out that it is hard to find gdb backtraces on musl, because
corresponding section is not highlighted. let us fix that

Ilya Shipitsin (2):
  CI: musl: highlight section if there are coredumps
  CI: musl: drop shopt in workflow invocation

 .github/workflows/musl.yml | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

-- 
2.41.0

[PATCH 1/2] CI: musl: highlight section if there are coredumps

[Ilya Shipitsin]

previously, section was collapsed, thus it was harder to find that
there's something to look at
---
 .github/workflows/musl.yml | 5 +
 1 file changed, 5 insertions(+)

diff --git a/.github/workflows/musl.yml b/.github/workflows/musl.yml
index 4017affb9..af358981f 100644
--- a/.github/workflows/musl.yml
+++ b/.github/workflows/musl.yml
@@ -40,12 +40,17 @@ jobs:
   - name: Show coredumps
 if: ${{ failure() && steps.vtest.outcome == 'failure' }}
 run: |
+  failed=false
   ls /tmp/core/
   for file in /tmp/core/core.*; do
+failed=true
 printf "::group::"
 gdb -ex 'thread apply all bt full' ./haproxy $file
 echo "::endgroup::"
   done
+  if [ "$failed" = true ]; then
+exit 1;
+  fi
   - name: Show results
 if: ${{ failure() }}
 run: |
-- 
2.41.0

[PATCH 2/2] CI: musl: drop shopt in workflow invocation

[Ilya Shipitsin]

"shopt" is bash specific, while musl uses bourne shell.

/__w/_temp/1b0f5f5d-c71b-4a66-8be3-e1fe51c10993.sh: line 7: shopt: not found
---
 .github/workflows/musl.yml | 1 -
 1 file changed, 1 deletion(-)

diff --git a/.github/workflows/musl.yml b/.github/workflows/musl.yml
index af358981f..930a22e01 100644
--- a/.github/workflows/musl.yml
+++ b/.github/workflows/musl.yml
@@ -60,4 +60,3 @@ jobs:
 cat $folder/LOG
 echo "::endgroup::"
   done
-  shopt -s nullglob
-- 
2.41.0

Re: [PATCH] CI: Update to actions/checkout@v4

[Willy Tarreau]

Hi Tim!

On Wed, Sep 06, 2023 at 04:57:29PM +0200, Tim Duesterhus wrote:
> No functional change, but we should keep this current.

Now merged, thank you!
Willy

[PATCH] CI: Update to actions/checkout@v4

[Tim Duesterhus]

No functional change, but we should keep this current.

see 5f4ddb54b05ae0355b1f64c22263a6bc381410df
---
 .github/workflows/aws-lc.yml   | 2 +-
 .github/workflows/codespell.yml| 2 +-
 .github/workflows/compliance.yml   | 2 +-
 .github/workflows/contrib.yml  | 2 +-
 .github/workflows/coverity.yml | 2 +-
 .github/workflows/cross-zoo.yml| 2 +-
 .github/workflows/fedora-rawhide.yml   | 2 +-
 .github/workflows/musl.yml | 2 +-
 .github/workflows/openssl-nodeprecated.yml | 2 +-
 .github/workflows/vtest.yml| 4 ++--
 .github/workflows/windows.yml  | 2 +-
 11 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/.github/workflows/aws-lc.yml b/.github/workflows/aws-lc.yml
index d884e3e79..b50292bb1 100644
--- a/.github/workflows/aws-lc.yml
+++ b/.github/workflows/aws-lc.yml
@@ -12,7 +12,7 @@ jobs:
   test:
 runs-on: ubuntu-latest
 steps:
-  - uses: actions/checkout@v3
+  - uses: actions/checkout@v4
   - name: Install VTest
 run: |
   scripts/build-vtest.sh
diff --git a/.github/workflows/codespell.yml b/.github/workflows/codespell.yml
index bacd87670..781aa8332 100644
--- a/.github/workflows/codespell.yml
+++ b/.github/workflows/codespell.yml
@@ -11,7 +11,7 @@ jobs:
   codespell:
 runs-on: ubuntu-latest
 steps:
-- uses: actions/checkout@v3
+- uses: actions/checkout@v4
 - uses: codespell-project/codespell-problem-matcher@v1
 - uses: codespell-project/actions-codespell@master
   with:
diff --git a/.github/workflows/compliance.yml b/.github/workflows/compliance.yml
index 509eaf841..caf9624a8 100644
--- a/.github/workflows/compliance.yml
+++ b/.github/workflows/compliance.yml
@@ -19,7 +19,7 @@ jobs:
   CC: gcc
   os: ubuntu-latest
 steps:
-- uses: actions/checkout@v3
+- uses: actions/checkout@v4
 - name: Install h2spec
   id: install-h2spec
   run: |
diff --git a/.github/workflows/contrib.yml b/.github/workflows/contrib.yml
index 99a1576d8..4e1474481 100644
--- a/.github/workflows/contrib.yml
+++ b/.github/workflows/contrib.yml
@@ -10,7 +10,7 @@ jobs:
   build:
 runs-on: ubuntu-latest
 steps:
-- uses: actions/checkout@v3
+- uses: actions/checkout@v4
 - name: Compile admin/halog/halog
   run: |
 make admin/halog/halog
diff --git a/.github/workflows/coverity.yml b/.github/workflows/coverity.yml
index ab4795e8c..0b3c4af91 100644
--- a/.github/workflows/coverity.yml
+++ b/.github/workflows/coverity.yml
@@ -17,7 +17,7 @@ jobs:
 runs-on: ubuntu-latest
 if: ${{ github.repository_owner == 'haproxy' }}
 steps:
-- uses: actions/checkout@v3
+- uses: actions/checkout@v4
 - name: Install apt dependencies
   run: |
 sudo apt-get update
diff --git a/.github/workflows/cross-zoo.yml b/.github/workflows/cross-zoo.yml
index f2c8d7ad8..d9864e298 100644
--- a/.github/workflows/cross-zoo.yml
+++ b/.github/workflows/cross-zoo.yml
@@ -97,7 +97,7 @@ jobs:
 sudo apt-get -yq --force-yes install \
 gcc-${{ matrix.platform.arch }} \
 ${{ matrix.platform.libs }}
-- uses: actions/checkout@v3
+- uses: actions/checkout@v4
 
 
 - name: install quictls
diff --git a/.github/workflows/fedora-rawhide.yml 
b/.github/workflows/fedora-rawhide.yml
index aa3abc585..8f2578154 100644
--- a/.github/workflows/fedora-rawhide.yml
+++ b/.github/workflows/fedora-rawhide.yml
@@ -17,7 +17,7 @@ jobs:
 container:
   image: fedora:rawhide
 steps:
-- uses: actions/checkout@v3
+- uses: actions/checkout@v4
 - name: Install dependencies
   run: |
 dnf -y install git pcre-devel zlib-devel pcre2-devel 'perl(FindBin)' 
perl-IPC-Cmd 'perl(File::Copy)' 'perl(File::Compare)' lua-devel socat findutils 
systemd-devel clang
diff --git a/.github/workflows/musl.yml b/.github/workflows/musl.yml
index 8eb8310fd..4017affb9 100644
--- a/.github/workflows/musl.yml
+++ b/.github/workflows/musl.yml
@@ -20,7 +20,7 @@ jobs:
 run: |
   ulimit -c unlimited
   echo '/tmp/core/core.%h.%e.%t' > /proc/sys/kernel/core_pattern
-  - uses: actions/checkout@v3
+  - uses: actions/checkout@v4
   - name: Install dependencies
 run: apk add gcc gdb make tar git python3 libc-dev linux-headers 
pcre-dev pcre2-dev openssl-dev lua5.3-dev grep socat curl musl-dbg lua5.3-dbg
   - name: Install VTest
diff --git a/.github/workflows/openssl-nodeprecated.yml 
b/.github/workflows/openssl-nodeprecated.yml
index e7f7ffaa5..a04c6cbfe 100644
--- a/.github/workflows/openssl-nodeprecated.yml
+++ b/.github/workflows/openssl-nodeprecated.yml
@@ -21,7 +21,7 @@ jobs:
   test:
 runs-on: ubuntu-latest
 steps:
-- uses: actions/checkout@v3
+- uses: actions/checkout@v4
 - name: Install VTest
   run: |
 scripts/build-vtest.sh
diff --git a/.github/workflows/vtest.yml 

Re: [PATCH 0/1] Introduce 'NOROOM' status for srv_state metric

[Willy Tarreau]

On Wed, Sep 06, 2023 at 12:44:56PM +, Cedric Paillet wrote:
> >I think that it might be more suitable to use the term "FULL"
> Ok, no problem with that. (Perhaps we can also rename server_has_room to 
> !server_is_full ?)

You're right, it might be clearer!

> > Also, a more accurate metric that is generally watched is the queue (both 
> > server and backend): 
> 
> My first use case is to detect/display one or more server(s) with problems,
> not to determine if the entire backend is undersized.
> As I understand it, if just one server in the pool is very slow (and the load
> balancing method is round-robin), the number of sessions on this server will
> increase until it reaches maxconn. At this juncture, the server will no
> longer be selected, and requests will be routed to the other servers. Then,
> no queue (either backend or server) will start to fill up, correct? But the
> slow server will cease receiving requests until its session count drops below
> maxconn, right?

Indeed, so actually you found a pretty valid use case to the edge case I
was seeing as rare enough not to be of much interest :-)

> The second use case, as you've outlined, is to detect if a backend is
> undersized. My understanding is that if the backend is "nearly" undersized,
> the first symptom will be some servers reporting "FULL". Only when ALL
> servers are reporting "FULL" will the backend queue start to grow, correct?

Yes that's it. For this specific use case, watching the queue is much
simpler and provides more than just a boolean. But to be clear, the
situation between one server full and all server full is very unstable,
as queuing is an exponential function of response time by definition,
so once a server is full, you can be certain that all other ones are
about to be full as well, and between the two you'll have a lot of
noise where servers are randomly full then ready depending on the
load. Let's say you're having 40 servers processing 1 requests
per second with a maxconn 50. That's a total capacity of 2000
outstanding requests (and as many concurrent connections). This
means requests will last on average 200ms at saturation. At 196 ms
response time you'll have 1960 outstanding requests, hence 49 per
server, thus no server marked full. At 200ms all will be full. At 198ms,
half of the servers will be marked full, and that changes for every
single request, i.e. 250 times per second per server. Now you can see
how sampling such a boolean only once every few seconds when it changes
250 times per second will be useless for scalability purposes, you're
just sampling a random value. On the opposite, retrieving the queue
length is more stable since it's for the whole backend and not just
per server, plus it reports more than a boolean.

Hoping this helps,
Willy

RE: [PATCH 0/1] Introduce 'NOROOM' status for srv_state metric

[Cedric Paillet]

Thanks for your review.

>I think that it might be more suitable to use the term "FULL"
Ok, no problem with that. (Perhaps we can also rename server_has_room to 
!server_is_full ?)


> Also, a more accurate metric that is generally watched is the queue (both 
> server and backend): 

My first use case is to detect/display one or more server(s) with problems, not 
to determine if the entire backend is undersized.
As I understand it, if just one server in the pool is very slow (and the load 
balancing method is round-robin), the number of sessions on this server will 
increase until it reaches maxconn. At this juncture, the server will no longer 
be selected, and requests will be routed to the other servers. Then, no queue 
(either backend or server) will start to fill up, correct? But the slow server 
will cease receiving requests until its session count drops below maxconn, 
right?

The second use case, as you've outlined, is to detect if a backend is 
undersized. My understanding is that if the backend is "nearly" undersized, the 
first symptom will be some servers reporting "FULL". Only when ALL servers are 
reporting "FULL" will the backend queue start to grow, correct?

Cédric


-Message d'origine-
De : Willy Tarreau  
Envoyé : mercredi 6 septembre 2023 10:36
À : Cedric Paillet 
Cc : haproxy@formilux.org
Objet : Re: [PATCH 0/1] Introduce 'NOROOM' status for srv_state metric

Hi Cedric,

On Tue, Sep 05, 2023 at 01:40:14PM +, Cedric Paillet wrote:
> We are using Prometheus to provide feedback to our users about the 
> status of backend servers. Currently, we have no means of informing 
> them if a server exceeds the maxconn limit, and consequently why it's 
> no longer receiving new requests.
> 
> Therefore, we would like to be able to display when a server surpasses 
> the maxconn limit and is in the "noroom" state. I have prepared a 
> patch specifically for Prometheus, but it might be better to include a 
> boolean directly in the server structure indicating whether the server 
> was considered to have no room the last time server_has_room was 
> called. However, this change seems to have a significant impact on other 
> parts of the code.

I think that it might be more suitable to use the term "FULL" that we already 
use for the listeners, and that also matches what's used at a few places in the 
doc in association with servers or backends being "full".

Also, a more accurate metric that is generally watched is the queue (both 
server and backend): instead of being a boolean, it directly indicates how many 
additional servers are needed to improve the processing time. Persistent 
requests are placed into the server's queue but all other requests go into the 
backend queue. So if you have a total capacity of 4 servers * 100 connections = 
400 outstanding requests, and you see a queue of 200, you know that you'd need 
two extra servers to process these without queuing, and you can even predict 
that the total processing time will decrease by 200*the average queue time, so 
this allows to even add just the required number of servers to keep response 
time below a certain limit.

The only case where a server will be full without having any queue is when the 
total number of outstanding requests on a backend is exactly equal to the sum 
of the servers' maxconn values, so as you see, the extra metric covers very 
little compared to the queue itself. But there might be good use cases for 
this, I'm not denying it, I just wanted to make sure that you're going to 
monitor what's really relevant for your use case ;-)

Willy

Re: [PATCH] BUILD: ssl: Build with new cryptographic library AWS-LC

[Илья Шипицин]

based on USE_OPENSSL_AWSLC quic may be enabled  ?

ср, 6 сент. 2023 г. в 14:26, William Lallemand :

> On Tue, Sep 05, 2023 at 11:56:26PM +, Hopkins, Andrew wrote:
> > I split up the remaining CI changes into 4 new attached patches. The
> > latest changes are still passing on my fork
> > https://github.com/andrewhop/haproxy/actions/runs/6090899582.
> >
>
> Thanks, I just merged them!
>
>
> > I was hoping to take advantage of the better HAProxy support in
> > AWS-LC's CI but I'm running into some issues in
> > https://github.com/aws/aws-lc/pull/1174 I was wondering if you had any
> > pointers of what to look at. I think this is CodeBuild specific issue
> > since the tests pass in HAProxy's CI and when I run AWS-LC's CI
> > locally. I just can't figure out what CodeBuild might be doing to mess
> > with the results.
> >
> > Looking at the log for mcli_start_progs.vtc the two sleep programs are
> > started as expected but the overall process returns the wrong exit
> > code (0x0 instead of 0x82). Does anything stand out to you as weird
> > looking?
> >
>
> I never used CodeBuild so I'm not aware on any timers or process
> limitation but that could be something like that.
>
> From what I understand from the trace, I think every processes received a
> SIGTERM. You can see 2 "Exiting Master process..." and the first one is
> before
> the "kill" from VTest which is suppose to send a SIGINT so it was probably
> sent
> outside the test.
>
> This test should finish like this:
>
> ***  h1debug|:MASTER.accept(0008)=000e from [127.0.0.1:41542]
> ALPN=
> ***  h1debug|:MASTER.srvcls[000e:]
>  h1CLI connection normally closed
> ***  h1CLI closing fd 9
>  h1CLI recv|#
>  
>  h1CLI recv|357949  master  0 [failed: 0]
>  0d00h00m00s 2.9-dev4-06d369-78
>  h1CLI recv|# workers
>  h1CLI recv|357955  worker  0
>  0d00h00m00s 2.9-dev4-06d369-78
>  h1CLI recv|# programs
>  h1CLI recv|357953  foo 0
>  0d00h00m00s -
>  h1CLI recv|357954  bar 0
>  0d00h00m00s -
>  h1CLI recv|
> ***  h1debug|0001:MASTER.clicls[:]
> ***  h1debug|0001:MASTER.closed[:]
>  h1CLI expect match ~ ".*foo.*
> .*bar.*
> "
> **   h1CLI ending
> **   h1Wait
> **   h1Stop HAproxy pid=357949
>  h1Kill(2)=0: Success
> ***  h1debug|[NOTICE]   (357949) : haproxy version is
> 2.9-dev4-06d369-78
> ***  h1debug|[NOTICE]   (357949) : path to executable is
> /home/wla/projects/haproxy/haproxy-community-maint/haproxy
> ***  h1debug|[WARNING]  (357949) : Exiting Master process...
> ***  h1debug|[ALERT](357949) : Current program 'foo' (357953)
> exited with code 130 (Interrupt)
> ***  h1debug|[ALERT](357949) : Current program 'bar' (357954)
> exited with code 130 (Interrupt)
>  dT0.076
> ***  h1debug|[ALERT](357949) : Current worker (357955) exited with
> code 130 (Interrupt)
> ***  h1debug|[WARNING]  (357949) : All workers exited. Exiting... (130)
>  dT0.077
>  h1STDOUT EOF
>  dT0.171
> **   h1WAIT4 pid=357949 status=0x8200 (user 0.058881 sys 0.026402)
> *top   RESETTING after reg-tests/mcli/mcli_start_progs.vtc
> **   h1Reset and free h1 haproxy -1
>  dT0.172
> **   s1Waiting for server (4/-1)
> *top   TEST reg-tests/mcli/mcli_start_progs.vtc completed
> *diag  0.0 /usr/bin/sleep
> #top  TEST reg-tests/mcli/mcli_start_progs.vtc passed (0.173)
> 0 tests failed, 0 tests skipped, 1 tests passed
>
>
> --
> William Lallemand
>

Re: [PATCH] BUILD: ssl: Build with new cryptographic library AWS-LC

[William Lallemand]

On Tue, Sep 05, 2023 at 11:56:26PM +, Hopkins, Andrew wrote:
> I split up the remaining CI changes into 4 new attached patches. The
> latest changes are still passing on my fork
> https://github.com/andrewhop/haproxy/actions/runs/6090899582.
> 

Thanks, I just merged them!


> I was hoping to take advantage of the better HAProxy support in
> AWS-LC's CI but I'm running into some issues in
> https://github.com/aws/aws-lc/pull/1174 I was wondering if you had any
> pointers of what to look at. I think this is CodeBuild specific issue
> since the tests pass in HAProxy's CI and when I run AWS-LC's CI
> locally. I just can't figure out what CodeBuild might be doing to mess
> with the results.
> 
> Looking at the log for mcli_start_progs.vtc the two sleep programs are
> started as expected but the overall process returns the wrong exit
> code (0x0 instead of 0x82). Does anything stand out to you as weird
> looking?
> 

I never used CodeBuild so I'm not aware on any timers or process
limitation but that could be something like that.

>From what I understand from the trace, I think every processes received a
SIGTERM. You can see 2 "Exiting Master process..." and the first one is before
the "kill" from VTest which is suppose to send a SIGINT so it was probably sent
outside the test.

This test should finish like this:

***  h1debug|:MASTER.accept(0008)=000e from [127.0.0.1:41542] 
ALPN=
***  h1debug|:MASTER.srvcls[000e:]
 h1CLI connection normally closed
***  h1CLI closing fd 9
 h1CLI recv|#   
  
 h1CLI recv|357949  master  0 [failed: 0]   0d00h00m00s 
2.9-dev4-06d369-78
 h1CLI recv|# workers
 h1CLI recv|357955  worker  0   0d00h00m00s 
2.9-dev4-06d369-78
 h1CLI recv|# programs
 h1CLI recv|357953  foo 0   0d00h00m00s 
-  
 h1CLI recv|357954  bar 0   0d00h00m00s 
-  
 h1CLI recv|
***  h1debug|0001:MASTER.clicls[:]
***  h1debug|0001:MASTER.closed[:]
 h1CLI expect match ~ ".*foo.*
.*bar.*
"
**   h1CLI ending
**   h1Wait
**   h1Stop HAproxy pid=357949
 h1Kill(2)=0: Success
***  h1debug|[NOTICE]   (357949) : haproxy version is 2.9-dev4-06d369-78
***  h1debug|[NOTICE]   (357949) : path to executable is 
/home/wla/projects/haproxy/haproxy-community-maint/haproxy
***  h1debug|[WARNING]  (357949) : Exiting Master process...
***  h1debug|[ALERT](357949) : Current program 'foo' (357953) exited 
with code 130 (Interrupt)
***  h1debug|[ALERT](357949) : Current program 'bar' (357954) exited 
with code 130 (Interrupt)
 dT0.076
***  h1debug|[ALERT](357949) : Current worker (357955) exited with code 
130 (Interrupt)
***  h1debug|[WARNING]  (357949) : All workers exited. Exiting... (130)
 dT0.077
 h1STDOUT EOF
 dT0.171
**   h1WAIT4 pid=357949 status=0x8200 (user 0.058881 sys 0.026402)
*top   RESETTING after reg-tests/mcli/mcli_start_progs.vtc
**   h1Reset and free h1 haproxy -1
 dT0.172
**   s1Waiting for server (4/-1)
*top   TEST reg-tests/mcli/mcli_start_progs.vtc completed
*diag  0.0 /usr/bin/sleep
#top  TEST reg-tests/mcli/mcli_start_progs.vtc passed (0.173)
0 tests failed, 0 tests skipped, 1 tests passed


-- 
William Lallemand

Re: [PATCH 0/1] Introduce 'NOROOM' status for srv_state metric

[Willy Tarreau]

Hi Cedric,

On Tue, Sep 05, 2023 at 01:40:14PM +, Cedric Paillet wrote:
> We are using Prometheus to provide feedback to our users about the status of
> backend servers. Currently, we have no means of informing them if a server
> exceeds the maxconn limit, and consequently why it's no longer receiving new
> requests.
> 
> Therefore, we would like to be able to display when a server surpasses the
> maxconn limit and is in the "noroom" state. I have prepared a patch
> specifically for Prometheus, but it might be better to include a boolean
> directly in the server structure indicating whether the server was considered
> to have no room the last time server_has_room was called. However, this
> change seems to have a significant impact on other parts of the code.

I think that it might be more suitable to use the term "FULL" that we
already use for the listeners, and that also matches what's used at a
few places in the doc in association with servers or backends being
"full".

Also, a more accurate metric that is generally watched is the queue
(both server and backend): instead of being a boolean, it directly
indicates how many additional servers are needed to improve the
processing time. Persistent requests are placed into the server's
queue but all other requests go into the backend queue. So if you have
a total capacity of 4 servers * 100 connections = 400 outstanding
requests, and you see a queue of 200, you know that you'd need two extra
servers to process these without queuing, and you can even predict that
the total processing time will decrease by 200*the average queue time,
so this allows to even add just the required number of servers to keep
response time below a certain limit.

The only case where a server will be full without having any queue is
when the total number of outstanding requests on a backend is exactly
equal to the sum of the servers' maxconn values, so as you see, the
extra metric covers very little compared to the queue itself. But there
might be good use cases for this, I'm not denying it, I just wanted to
make sure that you're going to monitor what's really relevant for your
use case ;-)

Willy

[HAPPRIVACY #YXL-289-28996]: China FO bearing export

[HAProxy Privacy]

ash@fobearings,

Thank you for contacting us. This is an automated response confirming the 
receipt of your ticket. One of our agents will get back to you as soon as 
possible. For your records, the details of the ticket are listed below. When 
replying, please make sure that the ticket ID is kept in the subject line to 
ensure that your replies are tracked appropriately.

Ticket ID: YXL-289-28996
Subject: [EXTERNAL] China FO bearing export
Department: Privacy
Type: Issue

Kind regards,

HAProxy Technologies

[HAPPRIVACY #TKN-189-63462]: China FO bearing export

[HAProxy Privacy]

ash@fobearings,

Thank you for contacting us. This is an automated response confirming the 
receipt of your ticket. One of our agents will get back to you as soon as 
possible. For your records, the details of the ticket are listed below. When 
replying, please make sure that the ticket ID is kept in the subject line to 
ensure that your replies are tracked appropriately.

Ticket ID: TKN-189-63462
Subject: [EXTERNAL] China FO bearing export
Department: Privacy
Type: Issue

Kind regards,

HAProxy Technologies