stable-bot: WARNING: 7 bug fixes in queue for next release

[stable-bot]

Hi,

This is a friendly bot that watches fixes pending for the next haproxy-stable 
release!  One such e-mail is sent periodically once patches are waiting in the 
last maintenance branch, and an ideal release date is computed based on the 
severity of these fixes and their merge date.  Responses to this mail must be 
sent to the mailing list.

Last release 1.8.19 was issued on 2019/02/11.  There are currently 7 patches in 
the queue cut down this way:
- 2 MAJOR, first one merged on 2019/03/11
- 3 MEDIUM, first one merged on 2019/03/11
- 2 MINOR, first one merged on 2019/03/11

Thus the computed ideal release date for 1.8.20 would be 2019/03/25, which was 
within the last week.

The current list of patches in the queue is:
- MAJOR   : spoe: Fix initialization of thread-dependent fields
- MAJOR   : listener: Make sure the listener exist before using it.
- MEDIUM  : 51d: fix possible segfault on deinit_51degrees()
- MEDIUM  : threads/fd: do not forget to take into account epoll_fd/pipes
- MEDIUM  : logs: Only attempt to free startup_logs once.
- MINOR   : listener: keep accept rate counters accurate under saturation
- MINOR   : ssl: fix warning about ssl-min/max-ver support

---
The haproxy stable-bot is freely provided by HAProxy Technologies to help 
improve the quality of each HAProxy release.  If you have any issue with these 
emails or if you want to suggest some improvements, please post them on the 
list so that the solutions suiting the most users can be found.

stable-bot: INFO: 3 bug fixes in queue for next release

[stable-bot]

Hi,

This is a friendly bot that watches fixes pending for the next haproxy-stable 
release!  One such e-mail is sent periodically once patches are waiting in the 
last maintenance branch, and an ideal release date is computed based on the 
severity of these fixes and their merge date.  Responses to this mail must be 
sent to the mailing list.

Last release 1.9.5 was issued on 2019/03/19.  There are currently 3 patches in 
the queue cut down this way:
- 3 MEDIUM, first one merged on 2019/03/21

Thus the computed ideal release date for 1.9.6 would be 2019/04/20, which is in 
four weeks or less.

The current list of patches in the queue is:
- MEDIUM  : mux-h2: Don't bother keeping the h2s if detaching and nothing 
to send.
- MEDIUM  : mux-h2: Make sure we destroyed the h2s once shutr/shutw is done.
- MEDIUM  : mux-h2: Use the right list in h2_stop_senders().

---
The haproxy stable-bot is freely provided by HAProxy Technologies to help 
improve the quality of each HAProxy release.  If you have any issue with these 
emails or if you want to suggest some improvements, please post them on the 
list so that the solutions suiting the most users can be found.

Re: H2 Protocol Errors in HTX Mode (1.9.4 & 1.9.4-dev)

[Willy Tarreau]

Hi Luke,

On Sat, Mar 23, 2019 at 02:52:26PM +0100, Luke Seelenbinder wrote:
> Hi Willy,
> 
> I just upgraded to 1.9.5, and this bug is still present (but seems to be
> somewhat diminished). On 1.9.4, approximately 5 of these images failed to
> load, on 1.9.5, it's usually 1 or 2. So overall it seems there is
> improvement, but something is still a bit wonky. :)

Christopher spotted issues around abortonclose and managed to address
them in an elegant way. We'll merge them into 2.0-dev and keep an eye
on them before backporting them to the next 1.9. I think we're getting
closer to having all these issues addressed. Additionally Olivier also
fixed some nasty issues affecting the streams ordering on H2 which can
make some streams starve for a while as long as there is activity on
others. I don't know if that's your case, but it could be possible that
it contributes to the issue. I'll let you know once all this stuff is
backported to 1.9. If you're interested in testing 2.0-dev, I'll tell
you once everything is merged there so that you don't waste your time
testing code that doesn't contain the fix.

Cheers,
Willy

Re: DNS Resolver Issues

[PiBa-NL]

Hi Daniel, Baptiste,

@Daniel, can you remove the 'addr loadbalancer-internal.xxx.yyy' from 
the server check? It seems to me that that name is not being resolved by 
the 'resolvers'. And even if it would it would be kinda redundant as it 
is in the example as it is the same as the servername.?. Not sure how 
far below scenarios are all explained by this though..


@Baptiste, is it intentional that a wrong 'addr' dns name makes haproxy 
fail to start despite having the supposedly never failing 
'default-server init-addr last,libc,none' ? Is it possibly a good 
feature request to support re-resolving a dns name for the addr setting 
as well ?


Regards,
PiBa-NL (Pieter)

Op 21-3-2019 om 20:37 schreef Daniel Schneller:

Hi!

Thanks for the response. I had looked at the "hold" directives, but since they 
all seem to have reasonable defaults, I did not touch them.
I specified 10s explictly, but it did not make a difference.

I did some more tests, however, and it seems to have more to do with the number 
of responses for the initial(?) DNS queries.
Hopefully these three tables make sense and don't get mangled in the mail. The 
"templated"
proxy is defined via "server-template" with 3 "slots". The "regular" one just as 
"server".


Test 1: Start out  with both "valid" and "broken" DNS entries. Then comment 
out/add back
one at a time as described in (1)-(5).
Each time after changing /etc/hosts, restart dnsmasq and check haproxy via 
hatop.
Haproxy started fresh once dnsmasq was set up to (1).

|  state   state
 /etc/hosts |  regular templated
|-
(1) BRK|  UP/L7OK DOWN/L4TOUT
 VALID  |  MAINT/resolution
|  UP/L7OK
|

(2) BRK|  DOWN/L4TOUT DOWN/L4TOUT
 #VALID |  MAINT/resolution
|  MAINT/resolution
|
(3) #BRK   |  UP/L7OK UP/L7OK
 VALID  |  MAINT/resolution
|  MAINT/resolution
|
(4) BRK|  UP/L7OK UP/L7OK
 VALID  |  DOWN/L4TOUT
|  MAINT/resolution
|
(5) BRK|  DOWN/L4TOUT DOWN/L4TOUT
 #VALID |  MAINT/resolution
|  MAINT/resolution
   
This all looks normal and as expected. As soon as the "VALID" DNS entry is present, the

UP state follows within a few seconds.
   



Test 2: Start out "valid only" (1) and proceed as described in (2)-(5), again 
restarting
dnsmasq each time, and haproxy reloaded after dnsmasq was set up to (1).

|  state   state
 /etc/hosts |  regular templated
|
(1) #BRK   |  UP/L7OK MAINT/resolution
 VALID  |  MAINT/resolution
|  UP/L7OK
|
(2) BRK|  UP/L7OK DOWN/L4TOUT
 VALID  |  MAINT/resolution
|  UP/L7OK
|
(3) #BRK   |  UP/L7OK MAINT/resolution
 VALID  |  MAINT/resolution
|  UP/L7OK
|
(4) BRK|  UP/L7OK DOWN/L4TOUT
 VALID  |  MAINT/resolution
|  UP/L7OK
|
(5) BRK|  DOWN/L4TOUT DOWN/L4TOUT
 #VALID |  MAINT/resolution
|  MAINT/resolution
   
Everything good here, too. Adding the broken DNS entry does not bring the proxies down

until only the broken one is left.



Test 3: Start out "broken only" (1).
Again, same as before, haproxy restarted once dnsmasq was initialized to (1).

|  state   state
 /etc/hosts |  regular templated
|
(1) BRK|  DOWN/L4TOUT DOWN/L4TOUT
 #VALID |  MAINT/resolution
|  MAINT/resolution

Re: H2 Protocol Errors in HTX Mode (1.9.4 & 1.9.4-dev)

[Luke Seelenbinder]

Hi Willy,

I just upgraded to 1.9.5, and this bug is still present (but seems to be 
somewhat diminished). On 1.9.4, approximately 5 of these images failed to load, 
on 1.9.5, it's usually 1 or 2. So overall it seems there is improvement, but 
something is still a bit wonky. :)

Best,
Luke

—
Luke Seelenbinder
SermonAudio.com  | Senior Software Engineer


> On Mar 1, 2019, at 05:35, Willy Tarreau  wrote:
> 
> On Fri, Feb 22, 2019 at 01:35:19PM +0100, Luke Seelenbinder wrote:
>> Hi List,
>> 
>> We recently started using HAProxy to act as a first point of entry for most
>> of our traffic. We initially set it up with H2 + HTX frontend and H1.1
>> backend; however, this led to some strange behavior consistently reproducible
>> on one page.
>> 
>> Whenever we loaded this page
>> (https://www.sermonaudio.com/search.asp?currsection=new=MP4) a we
>> would get a series of H2 protocol errors (SPDY_PROTOCOL_ERROR, as reported by
>> Chrome) for images stored on media-cloud.sermonaudio.com (running on the same
>> server / ip as www.semronaudio.com). Disabling HTX fixed the problem. Our
>> configuration is quite simple, just a few ACLs for redirection and forcing
>> SSL. I initially thought the issue could be related to the number of objects
>> being loaded (just over 100), so I tuned h2.max-concurrent-streams, but that
>> had no effect. This is observable in both 1.9.4 and the latest (as of two
>> days ago) 1.9 git branch.
> 
> That's a bit weird. I don't see what could cause a decoding error in HTX
> that doesn't happen without. I'll check if I can spot different code paths
> between HTX and legacy that could cause such errors to be emitted. One
> thing that could happen would be if in one of the htx decoding functions
> we end up reading a wrong amount of data sometimes, causing the stream to
> be desynchronized, but I really don't see where that would happen.
> 
>> I did not observe the issue on any other page but my search was not
>> exhaustive. It also never occurs on an individual request when made with
>> curl, for example.
> 
> OK that's useful info, it avois the usual "strange, works for me" after the
> first succcessful test :-)
> 
>> I would be able to stand up a server that has the same configuration with HTX
>> enabled for testing, if that would be helpful, you would have to provide the
>> /etc/hosts entries, though. :)
> 
> Might be. Let me take a look at the code first. Maybe I'll ask you to
> retry with a different version or to try a patch.
> 
> Thanks,
> Willy

Re: [PATCH] MINOR: ssl: Add aes_gcm_dec converter

[Nenad Merdanovic]

Hey Willy,

On 3/23/2019 11:24 AM, Willy Tarreau wrote:

I'm not sure why this is needed, because my first impression was that if
this part can be an argument in the decode it ought to be one as well for
the encoder, but that's where my ignorance of crypto shines, as I understand
from your explanation that it's something the encryption returns instead. I
thought that we'd simply have aes_gcm_encrypt(args),aes_gcm_decrypt(args)
be an identity in fact, but don't worry, I'm not asking you to do magics!


You are correct though. To extend the example:

str(foo),aes_gcm_enc(txn.nonce,txn.key,txn.aead_tag),aes_gcm_dec(txn.nonce,txn.key, 
txn.aead_tag)


This will still return "foo", but the txn.aead_tag in the "enc" 
converter is filled with nothing in the beginning, the enc converter 
will actually set the value of that variable after computing it. Then it 
must be used in decryption. Or it can be sent along with the ciphertext 
and the nonce to any other decryption implemention, that's why we need 
the ability to have it in a variable so it can be sent in a header for 
example.


The AEAD tag is a MAC which confirms the integrity and authenticity of 
the ciphertext and any AAD (additional authenticated data), which may be 
sent in the clear.


Cheers,
Nenad

Re: [PATCH] MINOR: ssl: Add aes_gcm_dec converter

[Willy Tarreau]

Hi Nenad,

On Sat, Mar 23, 2019 at 10:48:35AM +0100, Nenad Merdanovic wrote:
> >CC  src/ssl_sock.o
> > src/ssl_sock.c: In function 'sample_conv_aes_gcm_dec':
> > src/ssl_sock.c:9166:27: error: 'EVP_CTRL_AEAD_SET_IVLEN' undeclared (first 
> > use in this function)
> > src/ssl_sock.c:9166:27: note: each undeclared identifier is reported only 
> > once for each function it appears in
> > src/ssl_sock.c:9191:27: error: 'EVP_CTRL_AEAD_SET_TAG' undeclared (first 
> > use in this function)
> > distcc[22896] ERROR: compile src/ssl_sock.c on localhost failed
> > make: *** [src/ssl_sock.o] Error 1
> > 
> > It's on openssl 1.1.1a.
> 
> Are you sure that's 1.1.1a? I can actually see a problem with 1.0.2 and
> lower, where these are differently named, but work the same, so I am going
> to send a fix for that. I have double-checked that it builds fine with 1.1.1
> (and should with 1.1.0):
> Built with OpenSSL version : OpenSSL 1.1.1a  20 Nov 2018
> Running on OpenSSL version : OpenSSL 1.1.1a  20 Nov 2018

Bah obviously you're right. I was building with distcc and my cross-compiler
whose sysroot has 1.0.2 but typed "openssl version" on the command line...
When built against 1.1.1a it builds fine.

> I will, for now, also disable this for BoringSSL and LibreSSL until I manage
> to test it.

OK thanks!

> > Do you think it would be easy enough to have the equivalent encrypt
> > version ? :-)  I suspect it's mostly a matter of using the Encrypt
> > variant of the EVP functions, but I could really be wrong. This
> > could even allow to encrypt cookies in response and decrypt them
> > back for example.
> 
> Yes, that's the plan. It's a tiny bit more complex as I still need to think
> about how best to "return" two values. Current idea is to have the converter
> output the ciphertext and then as a last argument I will pass the variable
> name in which the AEAD tag will be stored by the converter.

I'm not sure why this is needed, because my first impression was that if
this part can be an argument in the decode it ought to be one as well for
the encoder, but that's where my ignorance of crypto shines, as I understand
from your explanation that it's something the encryption returns instead. I
thought that we'd simply have aes_gcm_encrypt(args),aes_gcm_decrypt(args)
be an identity in fact, but don't worry, I'm not asking you to do magics!

Cheers,
Willy

[PATCH] MINOR: ssl: Add aes_gcm_dec converter

[Nenad Merdanovic]

The converter can be used to decrypt the raw byte input using the
AES-GCM algorithm, using provided nonce, key and AEAD tag. This can
be useful to decrypt encrypted cookies for example and make decisions
based on the content.
---
 doc/configuration.txt |  12 
 src/ssl_sock.c| 148 ++
 2 files changed, 160 insertions(+)

diff --git a/doc/configuration.txt b/doc/configuration.txt
index 88148c3b..3ffe3d2f 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -13205,6 +13205,18 @@ add()
   This prefix is followed by a name. The separator is a '.'. The name may only
   contain characters 'a-z', 'A-Z', '0-9', '.' and '_'.
 
+aes_gcm_dec(,,,)
+  Decrypts the raw byte input using the AES128-GCM, AES192-GCM or
+  AES256-GCM algorithm, depending on the  parameter. All other parameters
+  need to be base64 encoded and the returned result is in raw byte format.
+  If the  validation fails, the converter doesn't return any data.
+  The ,  and  can either be strings or variables. This
+  converter requires at least OpenSSL 1.0.1.
+
+  Example:
+http-response set-header X-Decrypted-Text %[var(txn.enc),\
+  aes_gcm_dec(128,txn.nonce,Zm9vb2Zvb29mb29wZm9vbw==,txn.aead_tag)]
+
 and()
   Performs a bitwise "AND" between  and the input value of type signed
   integer, and returns the result as an signed integer.  can be a
diff --git a/src/ssl_sock.c b/src/ssl_sock.c
index 138b1c58..87e648d5 100644
--- a/src/ssl_sock.c
+++ b/src/ssl_sock.c
@@ -72,6 +72,11 @@
 #define X509_getm_notAfter  X509_get_notAfter
 #endif
 
+#if (OPENSSL_VERSION_NUMBER < 0x101fL && !defined OPENSSL_IS_BORINGSSL && 
!defined LIBRESSL_VERSION_NUMBER)
+#define EVP_CTRL_AEAD_SET_IVLEN EVP_CTRL_GCM_SET_IVLEN
+#define EVP_CTRL_AEAD_SET_TAG   EVP_CTRL_GCM_SET_TAG
+#endif
+
 #include 
 #include 
 
@@ -118,6 +123,7 @@
 #include 
 #include 
 #include 
+#include 
 
 /* Warning, these are bits, not integers! */
 #define SSL_SOCK_ST_FL_VERIFY_DONE  0x0001
@@ -9075,6 +9081,138 @@ static int cli_parse_set_ocspresponse(char **args, char 
*payload, struct appctx
 
 }
 
+#if (OPENSSL_VERSION_NUMBER >= 0x1000100fL && !defined OPENSSL_IS_BORINGSSL && 
!defined LIBRESSL_VERSION_NUMBER)
+static inline int sample_conv_var2smp_str(const struct arg *arg, struct sample 
*smp)
+{
+   switch (arg->type) {
+   case ARGT_STR:
+   smp->data.type = SMP_T_STR;
+   smp->data.u.str = arg->data.str;
+   return 1;
+   case ARGT_VAR:
+   if (!vars_get_by_desc(>data.var, smp))
+   return 0;
+   if (!sample_casts[smp->data.type][SMP_T_STR])
+   return 0;
+   if (!sample_casts[smp->data.type][SMP_T_STR](smp))
+   return 0;
+   return 1;
+   default:
+   return 0;
+   }
+}
+
+static int check_aes_gcm(struct arg *args, struct sample_conv *conv,
+ const char *file, int line, 
char **err)
+{
+   switch(args[0].data.sint) {
+   case 128:
+   case 192:
+   case 256:
+   break;
+   default:
+   memprintf(err, "key size must be 128, 192 or 256 (bits).");
+   return 0;
+   }
+   /* Try to decode a variable. */
+   vars_check_arg([1], NULL);
+   vars_check_arg([2], NULL);
+   vars_check_arg([3], NULL);
+   return 1;
+}
+
+/* Arguements: AES size in bits, nonce, key, tag. The last three arguments are 
base64 encoded */
+static int sample_conv_aes_gcm_dec(const struct arg *arg_p, struct sample 
*smp, void *private)
+{
+   struct sample nonce, key, aead_tag;
+   struct buffer *smp_trash, *smp_trash_alloc;
+   EVP_CIPHER_CTX *ctx;
+   int dec_size, ret;
+
+   smp_set_owner(, smp->px, smp->sess, smp->strm, smp->opt);
+   if (!sample_conv_var2smp_str(_p[1], ))
+   return 0;
+
+   smp_set_owner(, smp->px, smp->sess, smp->strm, smp->opt);
+   if (!sample_conv_var2smp_str(_p[2], ))
+   return 0;
+
+   smp_set_owner(_tag, smp->px, smp->sess, smp->strm, smp->opt);
+   if (!sample_conv_var2smp_str(_p[3], _tag))
+   return 0;
+
+   smp_trash = get_trash_chunk();
+   smp_trash_alloc = alloc_trash_chunk();
+   if (!smp_trash_alloc)
+   return 0;
+
+   ctx = EVP_CIPHER_CTX_new();
+
+   if (!ctx)
+   goto err;
+
+   dec_size = base64dec(nonce.data.u.str.area, nonce.data.u.str.data, 
smp_trash->area, smp_trash->size);
+   if (dec_size < 0)
+   goto err;
+   smp_trash->data = dec_size;
+
+   /* Set cipher type and mode */
+   switch(arg_p[0].data.sint) {
+   case 128:
+   EVP_DecryptInit_ex(ctx, EVP_aes_128_gcm(), NULL, NULL, NULL);
+   break;
+   case 192:
+   EVP_DecryptInit_ex(ctx, EVP_aes_192_gcm(), NULL, NULL, NULL);
+

Re: [PATCH] MINOR: ssl: Add aes_gcm_dec converter

[Nenad Merdanovic]

Hey Willy,

On 3/22/2019 5:38 PM, Willy Tarreau wrote:

Hi Nenad,

On Fri, Mar 22, 2019 at 12:02:24PM +0100, Nenad Merdanovic wrote:

The converter can be used to decrypt the raw byte input using the
AES-GCM algorithm, using provided nonce, key and AEAD tag. This can
be useful to decrypt encrypted cookies for example and make decisions
based on the content.


Do you think it would be easy enough to have the equivalent encrypt
version ? :-)  I suspect it's mostly a matter of using the Encrypt
variant of the EVP functions, but I could really be wrong. This
could even allow to encrypt cookies in response and decrypt them
back for example.


Yes, that's the plan. It's a tiny bit more complex as I still need to 
think about how best to "return" two values. Current idea is to have the 
converter output the ciphertext and then as a last argument I will pass 
the variable name in which the AEAD tag will be stored by the converter.




The patch looks good as-is, I'm applying it right now.

Thank you,
Willy



Not so good it seems :) I will send a fix in a few minutes.

Regards,
Nenad

Re: [PATCH] MINOR: ssl: Add aes_gcm_dec converter

[Nenad Merdanovic]

Hello Willy,

On 3/22/2019 5:40 PM, Willy Tarreau wrote:

Hmmm sorry, but I'm getting this here :

   CC  src/ssl_sock.o
src/ssl_sock.c: In function 'sample_conv_aes_gcm_dec':
src/ssl_sock.c:9166:27: error: 'EVP_CTRL_AEAD_SET_IVLEN' undeclared (first use 
in this function)
src/ssl_sock.c:9166:27: note: each undeclared identifier is reported only once 
for each function it appears in
src/ssl_sock.c:9191:27: error: 'EVP_CTRL_AEAD_SET_TAG' undeclared (first use in 
this function)
distcc[22896] ERROR: compile src/ssl_sock.c on localhost failed
make: *** [src/ssl_sock.o] Error 1

It's on openssl 1.1.1a.


Are you sure that's 1.1.1a? I can actually see a problem with 1.0.2 and 
lower, where these are differently named, but work the same, so I am 
going to send a fix for that. I have double-checked that it builds fine 
with 1.1.1 (and should with 1.1.0):

Built with OpenSSL version : OpenSSL 1.1.1a  20 Nov 2018
Running on OpenSSL version : OpenSSL 1.1.1a  20 Nov 2018

I will, for now, also disable this for BoringSSL and LibreSSL until I 
manage to test it.


Regards,
Nenad