Re: Logging SSL pre-master-key

[Willy Tarreau]

Hi Patrick,

On Fri, Jun 16, 2017 at 09:36:30PM -0400, Patrick Hemmer wrote:
> The main reason I had for supporting the older code is that it seems
> many (most?) linux distros, such as the one we use (CentOS/7), still
> ship with 1.0.1 or 1.0.2. However since this is a minor change and a
> feature enhancement, I doubt this will get backported to 1.7, meaning
> we'll have to manually patch it into the version we use. And since we're
> doing that, we can just use the patch that supports older OpenSSL.

Yes in this case I think it makes sense. And given the proliferation
of *ssl implementations, I really prefer to avoid adding code touching
directly internal openssl stuff.

> Anyway, here is the updated patch with the support for <1.1.0 dropped,
> as well as the BoringSSL support Emmanuel requested.

OK cool.

> One other minor thing I was unsure about was the fetch name. Currently I
> have it as "ssl_fc_session_key", but "ssl_fc_session_master_key" might
> be more accurate. However this is rather verbose and would make it far
> longer than any other sample fetch name, and I was unsure if there were
> rules around the naming.

There are no particular rules but it's true that long names are painful,
especially for those with "fat fingers". I'd suggest that given that it
starts with "ssl_fc_" and "ssl_bc_", the context is already quite
restricted and you could probably call them "smk" or "smkey". One would
argue that looking at the doc will be necessary, but with long names you
also have to look at the doc to find how to spell them anyway, the
difference being that short names don't mangle your configs too much,
especially in logformat strings.

> +ssl_bc_session_key : binary
> +  Returns the SSL session master key of the back connection when the outgoing
> +  connection was made over an SSL/TLS transport layer. It is useful to 
> decrypt
> +  traffic sent using ephemeral ciphers.

Here it would be nice to add a short sentence like "Not available before
openssl 1.1.0" so that users don't waste too much time on something not
working.

>  static int
> +smp_fetch_ssl_fc_session_key(const struct arg *args, struct sample *smp, 
> const char *kw, void *private)
> +{
> +#if OPENSSL_VERSION_NUMBER >= 0x1010L || defined(OPENSSL_IS_BORINGSSL)

Here I'd put the ifdef around the whole function, like we have for
ALPN for example, so that there's no risk it could be used in a non-working
config.

> + struct connection *conn = objt_conn((kw[4] != 'b') ? smp->sess->origin :
> + smp->strm ? smp->strm->si[1].end : 
> NULL);
> +
> + SSL_SESSION *ssl_sess;
> + int data_len;
> + struct chunk *data;
> +
> + if (!conn || !conn->xprt_ctx || conn->xprt != _sock)
> + return 0;
> +
> + ssl_sess = SSL_get_session(conn->xprt_ctx);
> + if (!ssl_sess)
> + return 0;
> +
> + data = get_trash_chunk();
> + data_len = SSL_SESSION_get_master_key(ssl_sess, (unsigned char 
> *)data->str, data->size);
> + if (!data_len)
> + return 0;
> +
> + smp->flags = SMP_F_CONST;
> + smp->data.type = SMP_T_BIN;
> + data->len = data_len;

If you want, you can even get rid of your data_len variable by directly
assigning SSL_SESSION_get_master_key() to data->len.

> +static int
>  smp_fetch_ssl_fc_sni(const struct arg *args, struct sample *smp, const char 
> *kw, void *private)
>  {
>  #ifdef SSL_CTRL_SET_TLSEXT_HOSTNAME
> @@ -7841,6 +7875,7 @@ static struct sample_fetch_kw_list 
> sample_fetch_keywords = {ILH, {
>   { "ssl_bc_unique_id",   smp_fetch_ssl_fc_unique_id,   0,
>NULL,SMP_T_BIN,  SMP_USE_L5SRV },
>   { "ssl_bc_use_keysize", smp_fetch_ssl_fc_use_keysize, 0,
>NULL,SMP_T_SINT, SMP_USE_L5SRV },
>   { "ssl_bc_session_id",  smp_fetch_ssl_fc_session_id,  0,
>NULL,SMP_T_BIN,  SMP_USE_L5SRV },
> + { "ssl_bc_session_key", smp_fetch_ssl_fc_session_key, 0,
>NULL,SMP_T_BIN,  SMP_USE_L5SRV },

And then this line an be surrounded by #ifdef. The real benefit is that
it refuses to parse where it will not work.

Thanks,
Willy

等待是二三流的客户，主动才有优质客户

[looseekubrian]

我已邀请您填写以下表单：
等待是二三流的客户，主动才有优质客户

要填写此表单，请访问：
https://docs.google.com/forms/d/e/1FAIpQLSdmNJJUHNn76tPxcad7i5eUi-_Tp1mprTtchk2YYEfvoMjeJw/viewform?c=0w=1usp=mail_form_link

Google表单：创建调查问卷并分析调查结果。

Re: Logging SSL pre-master-key

[Patrick Hemmer]

On 2017/6/16 09:34, Willy Tarreau wrote:
> Hi Patrick,
>
> On Mon, Jun 12, 2017 at 07:31:36PM -0400, Patrick Hemmer wrote:
>> I patched my haproxy to add a ssl_fc_session_key fetch, and with the
>> value I was able to decrypt my test sessions encrypted with
>> TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256.
>>
>> Since the implementation was fairly easy, I've included a patch for it.
>> But I've never submitted anything before, so there's a good chance of
>> something being wrong.
> No problem, that's what public review is made for. BTW at first glance
> your patch looks clean ;-)
>
>> The only thing is that the function to do the extraction was added in
>> 1.1.0
>> (https://github.com/openssl/openssl/commit/858618e7e037559b75b0bfca4d30440f9515b888)
>> The underlying vars are still there, and when I looked have been there
>> since as early as I could find (going back to 1998). But I'm not sure
>> how you feel about extracting the values without the helper function.
> I'd then suggest to proceed differently (if that's OK for you), which
> is to only expose this sample fetch function in 1.1.0 and above. If
> you're fine with running on 1.1 you won't feel any difference. Others
> who don't need this sample fetch right now will not feel any risk of
> build problem.
The main reason I had for supporting the older code is that it seems
many (most?) linux distros, such as the one we use (CentOS/7), still
ship with 1.0.1 or 1.0.2. However since this is a minor change and a
feature enhancement, I doubt this will get backported to 1.7, meaning
we'll have to manually patch it into the version we use. And since we're
doing that, we can just use the patch that supports older OpenSSL.

Anyway, here is the updated patch with the support for <1.1.0 dropped,
as well as the BoringSSL support Emmanuel requested.

One other minor thing I was unsure about was the fetch name. Currently I
have it as "ssl_fc_session_key", but "ssl_fc_session_master_key" might
be more accurate. However this is rather verbose and would make it far
longer than any other sample fetch name, and I was unsure if there were
rules around the naming.

-Patrick
From 217f02d6cd39cb0f40cae74098acf3b586442194 Mon Sep 17 00:00:00 2001
From: Patrick Hemmer 
Date: Mon, 12 Jun 2017 18:03:48 -0400
Subject: [PATCH] MINOR: ssl: add fetch 'ssl_fc_session_key' and
 'ssl_bc_session_key'

These fetches return the SSL master key of the front/back connection.
This is useful to decrypt traffic encrypted with ephemeral ciphers.
---
 doc/configuration.txt | 10 ++
 src/ssl_sock.c| 36 
 2 files changed, 46 insertions(+)

diff --git a/doc/configuration.txt b/doc/configuration.txt
index 49bfd85..3b9d96e 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -13930,6 +13930,11 @@ ssl_bc_session_id : binary
   made over an SSL/TLS transport layer. It is useful to log if we want to know
   if session was reused or not.
 
+ssl_bc_session_key : binary
+  Returns the SSL session master key of the back connection when the outgoing
+  connection was made over an SSL/TLS transport layer. It is useful to decrypt
+  traffic sent using ephemeral ciphers.
+
 ssl_bc_use_keysize : integer
   Returns the symmetric cipher key size used in bits when the outgoing
   connection was made over an SSL/TLS transport layer.
@@ -14185,6 +14190,11 @@ ssl_fc_session_id : binary
   a server. It is important to note that some browsers refresh their session ID
   every few minutes.
 
+ssl_fc_session_key : binary
+  Returns the SSL session master key of the front connection when the incoming
+  connection was made over an SSL/TLS transport layer. It is useful to decrypt
+  traffic sent using ephemeral ciphers.
+
 ssl_fc_sni : string
   This extracts the Server Name Indication TLS extension (SNI) field from an
   incoming connection made via an SSL/TLS transport layer and locally
diff --git a/src/ssl_sock.c b/src/ssl_sock.c
index 3680515..73fbc31 100644
--- a/src/ssl_sock.c
+++ b/src/ssl_sock.c
@@ -6170,6 +6170,40 @@ smp_fetch_ssl_fc_session_id(const struct arg *args, 
struct sample *smp, const ch
 }
 
 static int
+smp_fetch_ssl_fc_session_key(const struct arg *args, struct sample *smp, const 
char *kw, void *private)
+{
+#if OPENSSL_VERSION_NUMBER >= 0x1010L || defined(OPENSSL_IS_BORINGSSL)
+   struct connection *conn = objt_conn((kw[4] != 'b') ? smp->sess->origin :
+   smp->strm ? smp->strm->si[1].end : 
NULL);
+
+   SSL_SESSION *ssl_sess;
+   int data_len;
+   struct chunk *data;
+
+   if (!conn || !conn->xprt_ctx || conn->xprt != _sock)
+   return 0;
+
+   ssl_sess = SSL_get_session(conn->xprt_ctx);
+   if (!ssl_sess)
+   return 0;
+
+   data = get_trash_chunk();
+   data_len = SSL_SESSION_get_master_key(ssl_sess, (unsigned char 
*)data->str, data->size);
+   if (!data_len)
+   return 0;
+
+   smp->flags = 

Re: Possible regression in 1.6.12

[Frederic Lecaille]

On 06/16/2017 03:20 PM, Willy Tarreau wrote:

On Fri, Jun 16, 2017 at 03:10:56PM +0200, Willy Tarreau wrote:

Hi Veiko,

On Fri, Jun 16, 2017 at 01:41:14PM +0300, Veiko Kukk wrote:

So I have more info on this now. Veiko, first, I'm assuming that your config
was using "resolvers dns_resolvers" on the "server" line, otherwise resolvers
are not used.


My real world configs use resolvers, but timeouts happen even when resolver
was not used anywhere.


Wow then that's super strange, as by definition they are not even triggered
in this case!


I'm just realizing that it's very similar to the bug that Fred fixed here :

https://www.mail-archive.com/haproxy@formilux.org/msg26040.html


Ah, yes I remember this one. Calling close() in place of fd_delete() 
(which calls close()) puts a big mess in the fd cache for pollers. So 
not only on BSD systems ;)

Re: [ANNOUNCE] haproxy-1.7.6

[Willy Tarreau]

On Fri, Jun 16, 2017 at 07:49:16AM -0700, Kevin McArthur wrote:
> Any chance of getting the SNI pass-through to verifyhost supported into the
> next release? Bit of a security issue..

Unfortunately it cannot be backported since it doesn't exist at all in
mainline. Someone has to figure out how to do it and to implement it
first before it has a chance to exist in a maintenance branch.

For the short term, I guess the easiest we could do would possibly be to
at least emit a warning when SNI is configured on a server with verifyhost,
indicating that it can represent a risk since the cert's names are not
checked against the ones in the SNI. Then we can remove the warning when
the check is implemented.

Just my 2 cents,
Willy

Re: Possible regression in 1.6.12

[Willy Tarreau]

Hi Aleks,

On Fri, Jun 16, 2017 at 05:00:19PM +0200, Aleksandar Lazic wrote:
> > I tested both with 1.7.5 and 1.7.6 and can confirm that 1.7.5 has same 
> > request timeout issues that 1.6.12 has, but 1.7.6 works properly.
> 
> So a perfect reason to stay on 1.7 branch ;-)

Yes but here it's not really honnest, it's just because we've just
released the 1.7 version fixing it and that the 1.6 one has not yet
been ;-)

Willy

Re: master-worker and seamless reload (bug)

[William Lallemand]

On Fri, Jun 16, 2017 at 05:28:51PM +0200, Emmanuel Hocdet wrote:
> Hi,
>

Hi Emmanuel,
 
> i try to play with that, but i’m a little confused with the behaviour.
> 
> In my test, i use alternatly haproxy upgrade and worker reload (via USR2)
> 
> start with upgrade:
> # /usr/sbin/haproxy -f /var/lib/haproxy/ssl/ssl.cfg -p 
> /var/run/haproxy_ssl.pid -D -W -n 131072 -L ssl_1 -x 
> /var/run/haproxy/ssl.sock1  -x /var/run/haproxy/ssl.sock2  -sf `cat 
> /var/run/haproxy_ssl.pid`
> [ALERT] 166/165607 (13616) : Current worker 13618 left with exit code 0
> [ALERT] 166/165607 (13616) : Current worker 13617 left with exit code 0
> [WARNING] 166/165607 (13616) : All workers are left. Leaving... (0)

I'm not sure of what you are trying to do, do you try to upgrade an HAProxy
which is not using the master worker mode to a master worker?

In master worker mode the reload is meant to be done only with the USR2 signal,
the binary will be reloaded upon this signal so you don't have to start another
process to upgrade the binary.

> # ps auxwww |grep ssl
> 
> root 13635  0.0  0.0  79132   776 pts/0S16:59   0:00 
> /usr/sbin/haproxy -f /var/lib/haproxy/ssl/ssl.cfg -p /var/run/haproxy_ssl.pid 
> -D -W -n 131072 -L ssl_1 -x /var/run/haproxy/ssl.sock1 -x 
> /var/run/haproxy/ssl.sock2 -sf 13617 13618
> haproxy  13636  0.0  0.0  79132   940 ?Ss   16:59   0:00 
> /usr/sbin/haproxy -f /var/lib/haproxy/ssl/ssl.cfg -p /var/run/haproxy_ssl.pid 
> -D -W -n 131072 -L ssl_1 -x /var/run/haproxy/ssl.sock1 -x 
> /var/run/haproxy/ssl.sock2 -sf 13617 13618
> haproxy  13637  0.0  0.0  79132   940 ?Ss   16:59   0:00 
> /usr/sbin/haproxy -f /var/lib/haproxy/ssl/ssl.cfg -p /var/run/haproxy_ssl.pid 
> -D -W -n 131072 -L ssl_1 -x /var/run/haproxy/ssl.sock1 -x 
> /var/run/haproxy/ssl.sock2 -sf 13617 13618
> 
> seems ok, try to reload via USR2 on master.
> first note: the pid of master is not log in /var/run (only childrens) and i 
> don’t see any option to set it in a file (to use it in a script)
> 

That's right, it was one of the probable evolution. I will probably log only
the pid of the master in future patches.

> # kill -USR2 13635
> # [WARNING] 166/165947 (13635) : Reexecuting Master process
> [WARNING] 166/170818 (13635) : Former worker 13636 left with exit code 0
> [WARNING] 166/170818 (13635) : Former worker 13637 left with exit code 0
> 

Looks like a normal behavior.

> # ps auxwww |grep ssl
> root 13635  0.0  2.1  79132 44424 pts/0S16:59   0:00 
> /usr/sbin/haproxy -f /var/lib/haproxy/ssl/ssl.cfg -p /var/run/haproxy_ssl.pid 
> -D -W -n 131072 -L ssl_1 -x /var/run/haproxy/ssl.sock2 -x 
> /var/run/haproxy/ssl.sock2 -sf 13636 13637 13617 13618
> haproxy  13652  0.0  0.0  79132  1188 ?Ss   17:08   0:00 
> /usr/sbin/haproxy -f /var/lib/haproxy/ssl/ssl.cfg -p /var/run/haproxy_ssl.pid 
> -D -W -n 131072 -L ssl_1 -x /var/run/haproxy/ssl.sock2 -x 
> /var/run/haproxy/ssl.sock2 -sf 13636 13637 13617 13618
> haproxy  13653  0.0  0.0  79132  1176 ?Ss   17:08   0:00 
> /usr/sbin/haproxy -f /var/lib/haproxy/ssl/ssl.cfg -p /var/run/haproxy_ssl.pid 
> -D -W -n 131072 -L ssl_1 -x /var/run/haproxy/ssl.sock2 -x 
> /var/run/haproxy/ssl.sock2 -sf 13636 13637 13617 13618
> root 13655  0.0  0.0  11124   696 pts/0S+   17:08   0:00 grep ssl
> 
> I now see  -x /var/run/haproxy/ssl.sock2 twice (sock1 is lost) and -sf have 4 
> pids instead of 2.
> 

Well, that's a bug, the code of the mworker doesn't manage several -x. 

However you don't need to use several unix sockets, the fd of all listeners are
exposed in any process using the "expose-fd listeners" keyword, even if the
listeners are bind on another process.

The master worker reexec the process adding a -x with what it found in the
configuration, you don't have to start it with -x at startup.

Regarding the PID, I think you have this behavior because you started the
daemon mode with -sf. 

> one last:
> # haproxy -x
> Segmentation fault

I'll fix that.

> 
> ++
> Manu
> 

Thanks for the tests!

-- 
William Lallemand

master-worker and seamless reload (bug)

[Emmanuel Hocdet]

Hi,

i try to play with that, but i’m a little confused with the behaviour.

In my test, i use alternatly haproxy upgrade and worker reload (via USR2)

start with upgrade:
# /usr/sbin/haproxy -f /var/lib/haproxy/ssl/ssl.cfg -p /var/run/haproxy_ssl.pid 
-D -W -n 131072 -L ssl_1 -x /var/run/haproxy/ssl.sock1  -x 
/var/run/haproxy/ssl.sock2  -sf `cat /var/run/haproxy_ssl.pid`
[ALERT] 166/165607 (13616) : Current worker 13618 left with exit code 0
[ALERT] 166/165607 (13616) : Current worker 13617 left with exit code 0
[WARNING] 166/165607 (13616) : All workers are left. Leaving... (0)
# ps auxwww |grep ssl

root 13635  0.0  0.0  79132   776 pts/0S16:59   0:00 
/usr/sbin/haproxy -f /var/lib/haproxy/ssl/ssl.cfg -p /var/run/haproxy_ssl.pid 
-D -W -n 131072 -L ssl_1 -x /var/run/haproxy/ssl.sock1 -x 
/var/run/haproxy/ssl.sock2 -sf 13617 13618
haproxy  13636  0.0  0.0  79132   940 ?Ss   16:59   0:00 
/usr/sbin/haproxy -f /var/lib/haproxy/ssl/ssl.cfg -p /var/run/haproxy_ssl.pid 
-D -W -n 131072 -L ssl_1 -x /var/run/haproxy/ssl.sock1 -x 
/var/run/haproxy/ssl.sock2 -sf 13617 13618
haproxy  13637  0.0  0.0  79132   940 ?Ss   16:59   0:00 
/usr/sbin/haproxy -f /var/lib/haproxy/ssl/ssl.cfg -p /var/run/haproxy_ssl.pid 
-D -W -n 131072 -L ssl_1 -x /var/run/haproxy/ssl.sock1 -x 
/var/run/haproxy/ssl.sock2 -sf 13617 13618

seems ok, try to reload via USR2 on master.
first note: the pid of master is not log in /var/run (only childrens) and i 
don’t see any option to set it in a file (to use it in a script)

# kill -USR2 13635
# [WARNING] 166/165947 (13635) : Reexecuting Master process
[WARNING] 166/170818 (13635) : Former worker 13636 left with exit code 0
[WARNING] 166/170818 (13635) : Former worker 13637 left with exit code 0

# ps auxwww |grep ssl
root 13635  0.0  2.1  79132 44424 pts/0S16:59   0:00 
/usr/sbin/haproxy -f /var/lib/haproxy/ssl/ssl.cfg -p /var/run/haproxy_ssl.pid 
-D -W -n 131072 -L ssl_1 -x /var/run/haproxy/ssl.sock2 -x 
/var/run/haproxy/ssl.sock2 -sf 13636 13637 13617 13618
haproxy  13652  0.0  0.0  79132  1188 ?Ss   17:08   0:00 
/usr/sbin/haproxy -f /var/lib/haproxy/ssl/ssl.cfg -p /var/run/haproxy_ssl.pid 
-D -W -n 131072 -L ssl_1 -x /var/run/haproxy/ssl.sock2 -x 
/var/run/haproxy/ssl.sock2 -sf 13636 13637 13617 13618
haproxy  13653  0.0  0.0  79132  1176 ?Ss   17:08   0:00 
/usr/sbin/haproxy -f /var/lib/haproxy/ssl/ssl.cfg -p /var/run/haproxy_ssl.pid 
-D -W -n 131072 -L ssl_1 -x /var/run/haproxy/ssl.sock2 -x 
/var/run/haproxy/ssl.sock2 -sf 13636 13637 13617 13618
root 13655  0.0  0.0  11124   696 pts/0S+   17:08   0:00 grep ssl

I now see  -x /var/run/haproxy/ssl.sock2 twice (sock1 is lost) and -sf have 4 
pids instead of 2.

one last:
# haproxy -x
Segmentation fault

++
Manu

Re: Possible regression in 1.6.12

[Aleksandar Lazic]

Hi Veiko Kukk,

Veiko Kukk wrote on 16.06.2017:

> On 16/06/17 16:20, Willy Tarreau wrote:
>>
>> I'm just realizing that it's very similar to the bug that Fred fixed here :
>>
>> https://www.mail-archive.com/haproxy@formilux.org/msg26040.html
>>
>> Here it hanged on the unix socket but the problem sounds very similar. The
>> fix has not been backported to 1.6 yet, but could you just confirm that
>> 1.7.5 presents the same issue and that 1.7.6 fixes it ? We'll issue 1.6.13
>> ASAP with this fix then.

> I tested both with 1.7.5 and 1.7.6 and can confirm that 1.7.5 has same 
> request timeout issues that 1.6.12 has, but 1.7.6 works properly.

So a perfect reason to stay on 1.7 branch ;-)

> Best regards,
> Veiko

-- 
Best Regards
Aleks

Re: [ANNOUNCE] haproxy-1.7.6

[Kevin McArthur]

Any chance of getting the SNI pass-through to verifyhost supported into 
the next release? Bit of a security issue..


--

Kevin
On 2017-06-16 6:31 AM, William Lallemand wrote:

Hi,

HAProxy 1.7.6 was released on 2017/06/16. It added 37 new commits
after version 1.7.5.

As you may know, I'm now part of the stable release team of HAProxy along
with Willy and Cyril.

This is my first stable release which fixes a few major bugs:

- Olivier fixed a hang reported on FreeBSD. HAProxy was relying on an undefined
behavior in C to compute the timer which lead to various hangs every 49.7 days.
We now use the -fwrapv flag at compilation time to force the behavior of the
compiler. Binaries compiled with clang are more suited to be impacted by this
bug.

- Fred fixed a hang which is related to the DNS polling system. The fd of the
resolver was not unregistered but closed which lead to a hang of any new
connection using the same fd number.

- Willy fixed a runtime segfault caused by cookies and tarpit rules.

- Fred fixed a segfault occuring upon reload when parsing a server state file
in the case one of the servers was deleted from the configuration file.

Please find the usual URLs below :
Site index   : http://www.haproxy.org/
Discourse: http://discourse.haproxy.org/
Sources  : http://www.haproxy.org/download/1.7/src/
Git repository   : http://git.haproxy.org/git/haproxy-1.7.git/
Git Web browsing : http://git.haproxy.org/?p=haproxy-1.7.git
Changelog: http://www.haproxy.org/download/1.7/src/CHANGELOG
Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/


---
Complete changelog :
Adam Spiers (1):
   DOC: stick-table is available in frontend sections

Andrew Rodland (1):
   BUG/MINOR: hash-balance-factor isn't effective in certain circumstances

Christopher Faulet (4):
   BUG/MINOR: http: Fix conditions to clean up a txn and to handle the next 
request
   BUG/MINOR: buffers: Fix bi/bo_contig_space to handle full buffers
   BUG/MINOR: acls: Set the right refflag when patterns are loaded from a 
map
   BUG/MINOR: http/filters: Be sure to wait if a filter loops in 
HTTP_MSG_ENDING

Frédéric Lécaille (5):
   BUG/MINOR: dns: Wrong address family used when creating IPv6 sockets.
   BUG/MINOR: server: missing default server 'resolvers' setting 
duplication.
   BUG/MAJOR: dns: Broken kqueue events handling (BSD systems).
   BUG/MEDIUM: peers: Peers CLOSE_WAIT issue.
   BUG/MAJOR: server: Segfault after parsing server state file.

Glenn Strauss (2):
   DOC: update sample code for PROXY protocol
   DOC: mention lighttpd 1.4.46 implements PROXY

Jarno Huuskonen (4):
   DOC: changed "block"(deprecated) examples to http-request deny
   DOC: add few comments to examples.
   DOC: add layer 4 links/cross reference to "block" keyword.
   DOC: errloc/errorloc302/errorloc303 missing status codes.

Jim Freeman (1):
   CLEANUP: logs: typo: simgle => single

Lukas Tribus (1):
   DOC: update RFC references

Nan Liu (1):
   BUG/MINOR: Makefile: fix compile error with USE_LUA=1 in ubuntu16.04

Olivier Houchard (2):
   BUG/MAJOR: Use -fwrapv.
   BUG/MINOR: server: don't use "proxy" when px is really meant.

Thierry FOURNIER (3):
   BUG/MEDIUM: lua: memory leak
   MINOR/DOC: lua: just precise one thing
   BUG/MEDIUM: lua: segfault if a converter or a sample doesn't return 
anything

Willy Tarreau (12):
   BUG/MINOR: config: missing goto out after parsing an incorrect ACL 
character
   BUG/MINOR: arg: don't try to add an argument on failed memory allocation
   BUG/MEDIUM: arg: ensure that we properly unlink unresolved arguments on 
error
   BUG/MEDIUM: acl: don't free unresolved args in prune_acl_expr()
   MINOR: lua: ensure the memory allocator is used all the time
   BUG/MEDIUM: acl: proprely release unused args in prune_acl_expr()
   MEDIUM: config: don't check config validity when there are fatal errors
   BUG/MINOR: checks: don't send proxy protocol with agent checks
   BUG/MAJOR: http: call manage_client_side_cookies() before erasing the 
buffer
   BUG/MEDIUM: unix: never unlink a unix socket from the file system
   scripts: create-release pass -n to tail
   SCRIPTS: create-release: enforce GIT_COMMITTER_{NAME|EMAIL} validity

Re: SD Termination state after upgrade from 1.5.12 to 1.7.3

[Christopher Faulet]

Le 16/06/2017 à 13:29, Juan Pablo Mora a écrit :

Linux version:

Red Hat Enterprise Linux Server release 5.11 (Tikanga)

Linux dpoweb08 2.6.18-417.el5 #1 SMP Sat Nov 19 14:54:59 EST 2016 x86_64 
x86_64 x86_64 GNU/Linux


HAProxy versión: 1.7.5

Summary:

After upgrading to HAProxy 1.7.5, requests with "SD" status in logs have 
started to appear in one webservice. There have been no changes to the 
webservice we invoke. The log shows:


Jun 16 12:41:06 localhost.lognet.pre.logalty.es haproxy[17315]: 
172.31.2.70:59365 [16/Jun/2017:12:41:06.890] HTTP_INTERNO BUS/BUS1 
1/0/0/65/75 200 225565 390 - - SD-- 12/2/0/0/0 0/0 {|Keep-Alive|174|} 
{|close|} "POST 
/lgt/lgtbus/rest/private/receiverServiceLight/getSignedBinaryContent 
HTTP/1.1"


The same POST in HAProxy 1.5.12 ends with “ 200 “ and ““.



Hi,

Bernard McCormack has already reported the same issue. I'm on it. I 
approximately identified the problem but I need more time to fully 
understand it and to find the best way to fix it.


The bug occurs where we are waiting the end of the response to 
synchronize the request and the response. The close on the server side 
is detected as an error when there is no content-length nor 
transfer-encoding header. But it is a bit tricky because it is partly a 
timing issue. Depending when the connection close is catched, an error 
is triggered or not.


So I'm on it. Stay tuned.
--
Christopher Faulet

Re: Logging SSL pre-master-key

[Willy Tarreau]

Hi Patrick,

On Mon, Jun 12, 2017 at 07:31:36PM -0400, Patrick Hemmer wrote:
> I patched my haproxy to add a ssl_fc_session_key fetch, and with the
> value I was able to decrypt my test sessions encrypted with
> TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256.
> 
> Since the implementation was fairly easy, I've included a patch for it.
> But I've never submitted anything before, so there's a good chance of
> something being wrong.

No problem, that's what public review is made for. BTW at first glance
your patch looks clean ;-)

> The only thing is that the function to do the extraction was added in
> 1.1.0
> (https://github.com/openssl/openssl/commit/858618e7e037559b75b0bfca4d30440f9515b888)
> The underlying vars are still there, and when I looked have been there
> since as early as I could find (going back to 1998). But I'm not sure
> how you feel about extracting the values without the helper function.

I'd then suggest to proceed differently (if that's OK for you), which
is to only expose this sample fetch function in 1.1.0 and above. If
you're fine with running on 1.1 you won't feel any difference. Others
who don't need this sample fetch right now will not feel any risk of
build problem.

Cheers,
Willy

[ANNOUNCE] haproxy-1.7.6

[William Lallemand]

Hi,

HAProxy 1.7.6 was released on 2017/06/16. It added 37 new commits
after version 1.7.5.

As you may know, I'm now part of the stable release team of HAProxy along
with Willy and Cyril.

This is my first stable release which fixes a few major bugs:

- Olivier fixed a hang reported on FreeBSD. HAProxy was relying on an undefined
behavior in C to compute the timer which lead to various hangs every 49.7 days.
We now use the -fwrapv flag at compilation time to force the behavior of the
compiler. Binaries compiled with clang are more suited to be impacted by this
bug.

- Fred fixed a hang which is related to the DNS polling system. The fd of the
resolver was not unregistered but closed which lead to a hang of any new
connection using the same fd number. 

- Willy fixed a runtime segfault caused by cookies and tarpit rules.

- Fred fixed a segfault occuring upon reload when parsing a server state file
in the case one of the servers was deleted from the configuration file.

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Sources  : http://www.haproxy.org/download/1.7/src/
   Git repository   : http://git.haproxy.org/git/haproxy-1.7.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-1.7.git
   Changelog: http://www.haproxy.org/download/1.7/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/


---
Complete changelog :
Adam Spiers (1):
  DOC: stick-table is available in frontend sections

Andrew Rodland (1):
  BUG/MINOR: hash-balance-factor isn't effective in certain circumstances

Christopher Faulet (4):
  BUG/MINOR: http: Fix conditions to clean up a txn and to handle the next 
request
  BUG/MINOR: buffers: Fix bi/bo_contig_space to handle full buffers
  BUG/MINOR: acls: Set the right refflag when patterns are loaded from a map
  BUG/MINOR: http/filters: Be sure to wait if a filter loops in 
HTTP_MSG_ENDING

Frédéric Lécaille (5):
  BUG/MINOR: dns: Wrong address family used when creating IPv6 sockets.
  BUG/MINOR: server: missing default server 'resolvers' setting duplication.
  BUG/MAJOR: dns: Broken kqueue events handling (BSD systems).
  BUG/MEDIUM: peers: Peers CLOSE_WAIT issue.
  BUG/MAJOR: server: Segfault after parsing server state file.

Glenn Strauss (2):
  DOC: update sample code for PROXY protocol
  DOC: mention lighttpd 1.4.46 implements PROXY

Jarno Huuskonen (4):
  DOC: changed "block"(deprecated) examples to http-request deny
  DOC: add few comments to examples.
  DOC: add layer 4 links/cross reference to "block" keyword.
  DOC: errloc/errorloc302/errorloc303 missing status codes.

Jim Freeman (1):
  CLEANUP: logs: typo: simgle => single

Lukas Tribus (1):
  DOC: update RFC references

Nan Liu (1):
  BUG/MINOR: Makefile: fix compile error with USE_LUA=1 in ubuntu16.04

Olivier Houchard (2):
  BUG/MAJOR: Use -fwrapv.
  BUG/MINOR: server: don't use "proxy" when px is really meant.

Thierry FOURNIER (3):
  BUG/MEDIUM: lua: memory leak
  MINOR/DOC: lua: just precise one thing
  BUG/MEDIUM: lua: segfault if a converter or a sample doesn't return 
anything

Willy Tarreau (12):
  BUG/MINOR: config: missing goto out after parsing an incorrect ACL 
character
  BUG/MINOR: arg: don't try to add an argument on failed memory allocation
  BUG/MEDIUM: arg: ensure that we properly unlink unresolved arguments on 
error
  BUG/MEDIUM: acl: don't free unresolved args in prune_acl_expr()
  MINOR: lua: ensure the memory allocator is used all the time
  BUG/MEDIUM: acl: proprely release unused args in prune_acl_expr()
  MEDIUM: config: don't check config validity when there are fatal errors
  BUG/MINOR: checks: don't send proxy protocol with agent checks
  BUG/MAJOR: http: call manage_client_side_cookies() before erasing the 
buffer
  BUG/MEDIUM: unix: never unlink a unix socket from the file system
  scripts: create-release pass -n to tail
  SCRIPTS: create-release: enforce GIT_COMMITTER_{NAME|EMAIL} validity

-- 
William Lallemand

Re: Possible regression in 1.6.12

[Willy Tarreau]

On Fri, Jun 16, 2017 at 03:10:56PM +0200, Willy Tarreau wrote:
> Hi Veiko,
> 
> On Fri, Jun 16, 2017 at 01:41:14PM +0300, Veiko Kukk wrote:
> > > So I have more info on this now. Veiko, first, I'm assuming that your 
> > > config
> > > was using "resolvers dns_resolvers" on the "server" line, otherwise 
> > > resolvers
> > > are not used.
> > 
> > My real world configs use resolvers, but timeouts happen even when resolver
> > was not used anywhere.
> 
> Wow then that's super strange, as by definition they are not even triggered
> in this case!

I'm just realizing that it's very similar to the bug that Fred fixed here :

https://www.mail-archive.com/haproxy@formilux.org/msg26040.html

Here it hanged on the unix socket but the problem sounds very similar. The
fix has not been backported to 1.6 yet, but could you just confirm that
1.7.5 presents the same issue and that 1.7.6 fixes it ? We'll issue 1.6.13
ASAP with this fix then.

thanks,
Willy

Re: Possible regression in 1.6.12

[Willy Tarreau]

Hi Veiko,

On Fri, Jun 16, 2017 at 01:41:14PM +0300, Veiko Kukk wrote:
> > So I have more info on this now. Veiko, first, I'm assuming that your config
> > was using "resolvers dns_resolvers" on the "server" line, otherwise 
> > resolvers
> > are not used.
> 
> My real world configs use resolvers, but timeouts happen even when resolver
> was not used anywhere.

Wow then that's super strange, as by definition they are not even triggered
in this case!

(...)
> > BTW, (probably that it was just for illustration purpose), but please don't
> > use well-known services like google, yahoo or whatever for health checks. If
> > everyone does this, it will add a huge useless load to their servers.
> 
> It was just that anybody could use simple trimmed down configuration for
> quick testing. Real configuration has no need for having google.com as
> backend and is much more complex.

OK, I easily understand and appreciate this. For now I couldn't reproduce
the issue at all on 1.6.12 so that's why I suspected there were some other
differences.

> This exact configuration can be easily used to test 1.6.12 - a simple reload
> would cause two first google.com checks to fail with timouts. Also any
> requests against ssl-frontend will fail few first checks after reload.

I think I overlooked that part about the reload being needed for the issue
to happen. I'll try again with a reload.

Thanks for all the precisions,
Willy

Re: Possible regression in 1.6.12

[Veiko Kukk]

Hi, Willy

On 16/06/17 12:15, Willy Tarreau wrote:


So I have more info on this now. Veiko, first, I'm assuming that your config
was using "resolvers dns_resolvers" on the "server" line, otherwise resolvers
are not used.


My real world configs use resolvers, but timeouts happen even when 
resolver was not used anywhere. It is why I did not include resolvers in 
the example config backend server provided with initial report e-mail. 
When keeping only single server under resolvers section, I did not 
notice any timeouts. And it did not matter whether that single server 
was local or Google.



What I've seen when running your config here is that google responds both in
IPv4 and IPv6. And depending on your local network settings, if you can't
reach them over IPv6 after the address was updated, your connection might
get stuck waiting for the connect timeout to strike (10s in your conf,
multiplied by the number of retries). The way to address this is to add
"resolve-prefer ipv4" at the end of your server line, it will always pick
IPv4 addresses only.


We have 'resolve-prefer ipv4' enabled in real world configuration when 
resolver is used actually on 'server' line. We have disabled IPv6 for 
all our servers. Anyway - since timeouts happen even without using 
resolver anywhere, this must not be the cause of timeouts.



BTW, (probably that it was just for illustration purpose), but please don't
use well-known services like google, yahoo or whatever for health checks. If
everyone does this, it will add a huge useless load to their servers.


It was just that anybody could use simple trimmed down configuration for 
quick testing. Real configuration has no need for having google.com as 
backend and is much more complex.


This exact configuration can be easily used to test 1.6.12 - a simple 
reload would cause two first google.com checks to fail with timouts. 
Also any requests against ssl-frontend will fail few first checks after 
reload.


Regards,
Veiko

Re: Possible regression in 1.6.12

[Willy Tarreau]

On Fri, Jun 16, 2017 at 11:43:39AM +0200, Baptiste wrote:
> Guys,
> 
> I'll be able to have a look at this issue on Monday.
> 
> I quickly read the thread, and I feel it simply look like a configuration
> issue.
> Could you confirm what is the status of it?

At this point I think so as well, we'll wait for Veiko to double-check.

Thanks,
Willy

Re: Possible regression in 1.6.12

[Baptiste]

Guys,

I'll be able to have a look at this issue on Monday.

I quickly read the thread, and I feel it simply look like a configuration
issue.
Could you confirm what is the status of it?

Baptiste

Re: Logging SSL pre-master-key

[Emmanuel Hocdet]

Hi Patrick, Lukas

> Le 13 juin 2017 à 19:26, Lukas Tribus  a écrit :
> 
> Hi Patrick,
> 
> 
> Am 13.06.2017 um 01:31 schrieb Patrick Hemmer:
>> 
>> 
>> On 2017/6/12 15:14, Lukas Tribus wrote:
>>> Hello,
>>> 
>>> 
>>> Am 12.06.2017 um 19:35 schrieb Patrick Hemmer:
 Would we be able to get a new sample which provides the SSL session
 master-key?
 This is so that when performing packet captures with ephemeral ciphers
 (DHE), we can decrypt the traffic in the capture.
>>> There is no master key. What you need is the key for the symmetric
>>> crypto, and you cannot extract it from haproxy currently.
>>> 
>>> More importantly, OpenSSL implements this functionality only the master
>>> branch (see [1] and [2]), none of the release branches actually have
>>> this functionality.
>>> So we need OpenSSL to release a new branch with this functionality
>>> (1.1.1), we have to implement it in haproxy and then still it will only
>>> work for <=TLSv1.2.
>>> 
>>> TLSv1.3 will need additional secrets and a different key logging API [3].
>>> 
>>> 
>>> I suggest you use SSLKEYLOGFILE features in the browsers at this point,
>>> as the functionality is far from being ready for any OpenSSL based
>>> application.
>>> 
>>> 
>>> Regards,
>>> Lukas
>>> 
>>> [1]
>>> https://github.com/openssl/openssl/commit/2faa1b48fd6864f6bb8f992fd638378202fdd416
>>> [2]
>>> https://www.openssl.org/docs/manmaster/man3/SSL_CTX_set_keylog_callback.html
>>> [3] https://github.com/openssl/openssl/pull/2287
>>> 
>> 
>> Maybe there's some misunderstanding, because we seem to be talking
>> about different things, as there definitely is a master key.
>> 
>> I patched my haproxy to add a ssl_fc_session_key fetch, and with the
>> value I was able to decrypt my test sessions encrypted with
>> TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256.
> 
> We are talking about the same thing, I just find the term "master key"
> confusing, but I understand that is what everyone uses in this context.
> 
> 
> 
>> 
>> Since the implementation was fairly easy, I've included a patch for
>> it. But I've never submitted anything before, so there's a good chance
>> of something being wrong.
>> 
>> The only thing is that the function to do the extraction was added in
>> 1.1.0
>> (https://github.com/openssl/openssl/commit/858618e7e037559b75b0bfca4d30440f9515b888
>>  
>> )
> 
> Very nice, so the openssl features I mentioned are only necessary if we
> need callback functions (which is not the case), so we can already do
> this in 1.1.0.
> 
> 
> Patch needs review, CC +Emeric +Emmanuel.
> 
> 

Sounds good.

SSL_SESSION_get_master_key also exist in boringssl, the #if should be:
#if (OPENSSL_VERSION_NUMBER >= 0x1010fL) || defined(OPENSSL_IS_BORINGSSL)

Manu

Re: Possible regression in 1.6.12

[Willy Tarreau]

Hi guys,

On Thu, Jun 15, 2017 at 11:41:43AM +0200, Willy Tarreau wrote:
> On Thu, Jun 15, 2017 at 11:54:25AM +0300, Veiko Kukk wrote:
> > On 14/06/17 17:37, Willy Tarreau wrote:
> > > 
> > > Could you try to revert the attached patch which was backported to 1.6
> > > to fix an issue where nbproc and resolvers were incompatible ? To do
> > > that, please use "patch -Rp1 < foo.patch".
> > 
> > I have applied the patch. Now HAproxy working as in 1.6.11 version, no
> > requests time out.
> > 
> > > Also, have you noticed if your haproxy continues to work or if it loops
> > > at 100% CPU for example ?
> > 
> > No, there is no excessive CPU load.
> 
> OK thank you for this. Now I don't know what to think about it, because
> either it break a set of configs or another set :-/  Baptiste, could
> you please see how your change could have the effect described by Veiko ?

So I have more info on this now. Veiko, first, I'm assuming that your config
was using "resolvers dns_resolvers" on the "server" line, otherwise resolvers
are not used.

What I've seen when running your config here is that google responds both in
IPv4 and IPv6. And depending on your local network settings, if you can't
reach them over IPv6 after the address was updated, your connection might
get stuck waiting for the connect timeout to strike (10s in your conf,
multiplied by the number of retries). The way to address this is to add
"resolve-prefer ipv4" at the end of your server line, it will always pick
IPv4 addresses only.

Now as to why it changed after the fix, I'm speculating that it might be
that in fact till 1.6.11, most of the updates were lost, going to the wrong
process (due to nbproc being used), and that it caused the server's address
never to be changed. After the fix, the updates are properly delivered and
once you learn the IPv6 address the connections fail.

So in the end I think there's no regression, it's just that a bug was
silently hiding a config issue.

BTW, (probably that it was just for illustration purpose), but please don't
use well-known services like google, yahoo or whatever for health checks. If
everyone does this, it will add a huge useless load to their servers. And
they can cut at any time making your service fail. We're already experiencing
this on haproxy.org and sometimes I simply have to blacklist some source IP
addresses to make them realise that it's not the best idea.

Cheers,
Willy

Re: [PATCH] MEDIUM: ssl: allow haproxy to start without default certificate

[Emmanuel Hocdet]

> Le 15 juin 2017 à 16:42, Simos Xenitellis  a 
> écrit :
> 
> On Mon, Jun 12, 2017 at 5:21 PM, Emmanuel Hocdet  wrote:
>> In haproxy 1.8dev, default certificate can now be optional.
>> This patch allow that.
>> 
> 
> Thanks Manu for looking into this.
> 
> Here is my use-case:
> 
> 1. A "frontend" would bind on port 80 and then look whether a request
> is from Letsencrypt (URL: ~/.well-known/..). That is, an "http-01"
> challenge request.
> If so, it would forward the connection to a backend that deals with
> certificates (that backend initiated this request in the first place).
> If it is not an "http-01" challenge request, then it would redirect to https.
> 
> 2. Another frontend would bind to port 443, and the "bind" line would
> have a new keyword like "disable-if-no-certs".
> If there are no certs yet installed, haproxy would cancel out the
> whole frontend for port 443 and would not bind port 443.
> 
> 
> Ideally, this would be implemented cleanly if there was a way to simply 
> specify
> 
> use_frontend myhttps if { ssl_certs_exist }
> 
> Also, we could then specify to redirect to https (first frontend
> earlier ) if { ssl_certs_exist }.
> 

with this patch you will not need such complicated needs.
just do:
 bind :443 ssl strict-sni crt /my/cert/directory/

without this patch you need to have at least one certificate (fake or not)

> 
> For this to work, it would require:
> 
> 1. Addition of keyboard "use_frontend", just like "use_backend" exists.
> 2. HAProxy should set "ssl_certs_exist" when it loads up, depending on
> whether certificates have been found or not.
> 
> Simos
> 
> 
>> 
>>> Le 29 mai 2017 à 11:09, Emmanuel Hocdet  a écrit :
>>> 
>>> 
>>> Hi Simos,
>>> 
>>> The workaround is to have a default (fake) certificat in first and use « 
>>> strict-sni » parameter.
>>> 
>>> Manu
>>> 
 Le 22 mai 2017 à 10:28, Simos Xenitellis  a 
 écrit :
 
 Hi All,
 
 I am trying to automate some tasks with adding multiple https
 (LetsEncrypt) websites,
 and using HAProxy as a TLS Termination Proxy.
 
 The problem is that when you start off with an empty server, there are
 no certificates yet,
 and it is not possible to have "bind *:443 ssl crt
 /etc/haproxy/certs/..." in haproxy.cfg.
 
 LetsEncrypt can work with http, so it could easily use the "bind *:80"
 front-end in the beginning.
 
 Is there a way to express "If no certificates are found in
 /etc/haproxy/certs/, then do not bind *:443"?
 
 Simos
 
>>> 
>> 
>>