reverse proxy with dynamic servers without restart

[Michal Vala]

Hello,
I would like to do reverse proxy with haproxy so that part of the path
resolves to the server and I need to do it dynamically, servers are
created and destroyed on the fly. And I need to do the path rewrite so
the server part is removed from the path (I can do that with
http-request set-path %[path,regsub(^/ws-.+\//?,/)] on the backend).

for example 'http://example.com/server-123abc/api/users' requests
server 'server-123abc' on '/api/users'

Is it possible to do this completely dynamic, without knowing server
adresses ahead (we generate them dynamically with "random" names) AND
without restarting haproxy to load new configuration? I was thinking
about some regex matching, but I don't know how to do dynamic server
address and if it is somehow possible.

Thanks

-- 
Michal Vala
Software Engineer, Eclipse Che
Red Hat Czech

stable-bot: Bugfixes waiting for a release 2.1 (4), 2.0 (1)

[stable-bot]

Hi,

This is a friendly bot that watches fixes pending for the next haproxy-stable 
release!  One such e-mail is sent periodically once patches are waiting in the 
last maintenance branch, and an ideal release date is computed based on the 
severity of these fixes and their merge date.  Responses to this mail must be 
sent to the mailing list.


Last release 2.1.4 was issued on 2020-04-02.  There are currently 4 patches in 
the queue cut down this way:
- 4 MINOR, first one merged on 2020-04-02

Thus the computed ideal release date for 2.1.5 would be 2020-04-30, which was 
within the last week.

Last release 2.0.14 was issued on 2020-04-02.  There are currently 1 patches in 
the queue cut down this way:
- 1 MINOR, first one merged on 2020-04-02

Thus the computed ideal release date for 2.0.15 would be 2020-04-30, which was 
within the last week.

The current list of patches in the queue is:
 - 2.0, 2.1  - MINOR   : protocol_buffer: Wrong maximum 
shifting.
 - 2.1   - MINOR   : ssl: memleak of the struct 
cert_key_and_chain
 - 2.1   - MINOR   : ssl/cli: memory leak in 'set ssl cert'
 - 2.1   - MINOR   : connection: always send address-less 
LOCAL PROXY connections

-- 
The haproxy stable-bot is freely provided by HAProxy Technologies to help 
improve the quality of each HAProxy release.  If you have any issue with these 
emails or if you want to suggest some improvements, please post them on the 
list so that the solutions suiting the most users can be found.

Re: [PATCH] DOC: give a more accuration description of what check does

[Willy Tarreau]

On Tue, Apr 28, 2020 at 10:31:42PM +0200, Jerome Magnin wrote:
> >   - by default, checks are performed on the same address and port as
> > configured on the server, using the same encapsulation parameters
> > (SSL/TLS, SNI, proxy-protocol header etc).  If "port" or "addr"
> > directives are set, it is assumed that the server isn't checked on
> > the traffic port but on a proxy port that possibly uses a different
> > service. In this case, connection headers such as the proxy protocol
> > are only sent if "check-send-proxy" is set; SSL/TLS is only used if
> > "check-ssl" is used, and similarly the SNI used on the connection
> > will be defined by "check-sni". See also "check-ssl", "check-sni" etc.
> 
> This part is not entirely accurate in my experience:
>   - SNI configured with "sni" on server lines is not used with checks. One 
> must
> set "check-sni", if only because it's almost always 'sni ssl_fc_sni' or 
> 'sni hdr(host)' which has no meaning in the context of checks. I don't 
> think
> there are ways around this. 

That's a good point, that's forced by the fact that "sni" takes a dynamic
expression while checks cannot and must take a constant.

>   - the "alpn" setting of a server line is also not used for checks, one must
> define it with check-alpn. This will probably change soon now that haproxy
> can do h2 checks natively.

For this one I don't know :-)

OK I'm taking the patch as-is. If anyone who has experienced issues
configuring checks could have a look and see if all their issues are
properly addressed now, that would be great. Probably that we'd need
to have a "checks" section in the config manual at some point.

Thanks,
Willy

Re: [PATCH] DOC: give a more accuration description of what check does

[Jerome Magnin]

Hi Willy,

On Tue, Apr 28, 2020 at 05:46:08PM +0200, Willy Tarreau wrote:
> 
> Thanks for this, however I don't completely agree with the wording
> there (but I do agree that the current one is totally outdated).
> 
> The main point is that users don't configure "handshakes", they
> rather define transport protocols (typically SSL) so in the end
> this is not necessarily clearer this way and may even lead to new
> confusion regarding ssl vs check-ssl for example.
> 
> Probably that it would be better to take this opportunity to
> entirely revisit the description of the "check" keyword to cover
> the following points:

Thanks for the feedback. I've attached an updated patch to this email.

> 
>   - by default, checks are performed on the same address and port as
> configured on the server, using the same encapsulation parameters
> (SSL/TLS, SNI, proxy-protocol header etc).  If "port" or "addr"
> directives are set, it is assumed that the server isn't checked on
> the traffic port but on a proxy port that possibly uses a different
> service. In this case, connection headers such as the proxy protocol
> are only sent if "check-send-proxy" is set; SSL/TLS is only used if
> "check-ssl" is used, and similarly the SNI used on the connection
> will be defined by "check-sni". See also "check-ssl", "check-sni" etc.

This part is not entirely accurate in my experience:
  - SNI configured with "sni" on server lines is not used with checks. One must
set "check-sni", if only because it's almost always 'sni ssl_fc_sni' or 
'sni hdr(host)' which has no meaning in the context of checks. I don't think
there are ways around this. 
  - the "alpn" setting of a server line is also not used for checks, one must
define it with check-alpn. This will probably change soon now that haproxy
can do h2 checks natively.

Jérôme
>From 6a8e8ecfa1f6c9e8a4d1a5000296f650b45d9bdc Mon Sep 17 00:00:00 2001
From: Jerome Magnin 
Date: Sun, 26 Apr 2020 14:23:04 +0200
Subject: [PATCH] DOC: give a more accurate description of what check does

The documentation for check implies that without an application
level check configured, it only enables simple tcp checks. What it
actually does is verify that the configured transport layer is available,
and that optional application level checks succeed.
---
 doc/configuration.txt | 61 ++-
 1 file changed, 49 insertions(+), 12 deletions(-)

diff --git a/doc/configuration.txt b/doc/configuration.txt
index 3853d406f..12582d9f2 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -12520,18 +12520,55 @@ ca-file 
   server's certificate.
 
 check
-  This option enables health checks on the server. By default, a server is
-  always considered available. If "check" is set, the server is available when
-  accepting periodic TCP connections, to ensure that it is really able to serve
-  requests. The default address and port to send the tests to are those of the
-  server, and the default source is the same as the one defined in the
-  backend. It is possible to change the address using the "addr" parameter, the
-  port using the "port" parameter, the source address using the "source"
-  address, and the interval and timers using the "inter", "rise" and "fall"
-  parameters. The request method is define in the backend using the "httpchk",
-  "smtpchk", "mysql-check", "pgsql-check" and "ssl-hello-chk" options. Please
-  refer to those options and parameters for more information. See also
-  "no-check" option.
+  This option enables health checks on a server:
+- when not set, no health checking is performed, and the server is always
+  considered available.
+- when set and no other check method is configured, the server is 
considered
+  available when a connection can be established at the highest configured
+  transport layer. This means TCP by default, or SSL/TLS when "ssl" or
+  "check-ssl" are set, both possibly combined with connection prefixes such
+  as a PROXY protocol header when "send-proxy" or "check-send-proxy" are
+  set.
+- when set and an application-level health check is defined, the
+  application-level exchanges are performed on top of the configured
+  transport layer and the server is considered available if all of the
+  exchanges succeed.
+
+  By default, health checks are performed on the same address and port as
+  configured on the server, using the same encapsulation parameters (SSL/TLS,
+  proxy-protocol header, etc... ). It is possible to change the destination
+  address using "addr" and the port using "port". When done, it is assumed the
+  server isn't checked on the service port, and configured encapsulation
+  parameters are not reused. One must explicitely set "check-send-proxy" to 
send
+  connection headers, "check-ssl" to use SSL/TLS.
+
+  When "sni" or "alpn" are set on the server line, their value is not used for
+  health checks an

Re: [ANNOUNCE] haproxy-2.2-dev6

[Willy Tarreau]

On Tue, Apr 28, 2020 at 09:37:29PM +0200, William Dauchy wrote:
> On Tue, Apr 28, 2020 at 8:50 PM Willy Tarreau  wrote:
> > I've pushed the fix below now:
> >
> >   ca39747dcf ("BUG/MEDIUM: mux-h1: make sure we always have a timeout on 
> > front connections")
> >
> > Next time you rebuild, it could make sense to apply it (even on top of
> > your dev5) to improve the stability. If by any chance you retest dev6
> > I'm interested in knowing if you still get problems with it, of course :-)
> 
> Thanks for the followup, I will apply it on top of dev6 as I'm lazy
> adapting the patch on top of dev5 ;-)
> Our probing will quickly tell us the result!

Cool, thanks!

Willy

Re: [ANNOUNCE] haproxy-2.2-dev6

[William Dauchy]

On Tue, Apr 28, 2020 at 8:50 PM Willy Tarreau  wrote:
> So just to let you know, we could finally nail the issue down to a
> change I did on mux-h1 at the end of dev4 (so dev5 is also affected).
> Only front connections dying in a dirty way were affected (i.e. users
> disappearing from the net after they got a response, the timeout was
> not rearmed).

oh that odd, I based my statement on our automatic probing, so maybe
we have another issue.

> I'm not aware of anything past dev5 which could condition its
> occurrence. As such it's very likely that you have this bug as well
> as I don't see what could work around it, but that your workload makes
> you very unlikely to encounter it.
>
> I've pushed the fix below now:
>
>   ca39747dcf ("BUG/MEDIUM: mux-h1: make sure we always have a timeout on 
> front connections")
>
> Next time you rebuild, it could make sense to apply it (even on top of
> your dev5) to improve the stability. If by any chance you retest dev6
> I'm interested in knowing if you still get problems with it, of course :-)

Thanks for the followup, I will apply it on top of dev6 as I'm lazy
adapting the patch on top of dev5 ;-)
Our probing will quickly tell us the result!
-- 
William

Re: [ANNOUNCE] haproxy-2.2-dev6

[Willy Tarreau]

Hi William,

On Mon, Apr 27, 2020 at 08:34:48PM +0200, William Dauchy wrote:
> If this can help, the issue is possibly between dev5 and dev6. We have
> been following quite closely the dev releases, but we had to revert
> from dev6 to dev5 as it produced issues on our side - where it is
> perfectly running fine on dev5.

So just to let you know, we could finally nail the issue down to a
change I did on mux-h1 at the end of dev4 (so dev5 is also affected).
Only front connections dying in a dirty way were affected (i.e. users
disappearing from the net after they got a response, the timeout was
not rearmed).

I'm not aware of anything past dev5 which could condition its
occurrence. As such it's very likely that you have this bug as well
as I don't see what could work around it, but that your workload makes
you very unlikely to encounter it.

I've pushed the fix below now:

  ca39747dcf ("BUG/MEDIUM: mux-h1: make sure we always have a timeout on front 
connections")

Next time you rebuild, it could make sense to apply it (even on top of
your dev5) to improve the stability. If by any chance you retest dev6
I'm interested in knowing if you still get problems with it, of course :-)

Thanks,
Willy

Re: [PATCH] DOC: give a more accuration description of what check does

[Willy Tarreau]

Hi Jérôme,

On Sun, Apr 26, 2020 at 02:34:34PM +0200, Jerome Magnin wrote:
> The documentation for check implies that without an optional l7
> check configured, it enables simple tcp checks. What it actually
> does is check that every configured handshake on the server line
> can be established.

Thanks for this, however I don't completely agree with the wording
there (but I do agree that the current one is totally outdated).

The main point is that users don't configure "handshakes", they
rather define transport protocols (typically SSL) so in the end
this is not necessarily clearer this way and may even lead to new
confusion regarding ssl vs check-ssl for example.

Probably that it would be better to take this opportunity to
entirely revisit the description of the "check" keyword to cover
the following points:
  - without "check", no check is performed and the server is always
considered "up"

  - when "check" is specified and no check method is defined, the
server is considered "up" when a connection can be established
at the highest configured transport layer, which means bare TCP
by default, or SSL/TLS when "ssl" or "check-ssl" are involved,
both possibly combined with possible connection prefixes such as
a proxy protocol header

  - when "check" is specified and an application-level check method
is defined, then the application-level exchanges are performed on
top of the configured transport layer, all of which must succeed
for the server to be reported "up".

  - by default, checks are performed on the same address and port as
configured on the server, using the same encapsulation parameters
(SSL/TLS, SNI, proxy-protocol header etc).  If "port" or "addr"
directives are set, it is assumed that the server isn't checked on
the traffic port but on a proxy port that possibly uses a different
service. In this case, connection headers such as the proxy protocol
are only sent if "check-send-proxy" is set; SSL/TLS is only used if
"check-ssl" is used, and similarly the SNI used on the connection
will be defined by "check-sni". See also "check-ssl", "check-sni" etc.

> # this does a tcp connect + tls handshake
> backend foo
>   server s1 192.168.0.1;443 ssl check
   beware the semicolon here ^

Be careful below, some lines are slightly longer than the 80 chars limit.

> +  always considered available. If "check" is set, the server is considered
> +  available when all the handshakes configured on the server line can be
> +  established, or when an optional layer 7 health check succeeds. This is to
> +  ensure that the server is really able to handle requests. The default 
> address
> +  and port to send health checks to are those of the server, and the default
> +  source address is the same as the one defined in the backend. It is 
> possible
> +  to change the destination address using the "addr" parameter, the port 
> using
> +  the "port" parameter, the source address using the "source" parameter, and 
> the
> +  interval and timers using the "inter", "rise" and "fall" parameters. 
> Optional
> +  layer 7 checks can be configured with the "httpchk", "smtpchk", 
> "mysql-check",
> +  "pgsql-check" and "ssl-hello-check" options. When ssl is configured on the
> +  server line, "option tcp-check" and "tcp-check connect" can be used to 
> refrain
> +  from establishing the tls hanshake during health checks. Please refer to 
> those
> +  options and parameters for more information. See also "no-check" and 
> "check-ssl".

Thanks,
Willy

Re: [PATCH 0/1] *** Add Tu timer ***

[Willy Tarreau]

Hi Damien,

On Tue, Apr 28, 2020 at 12:09:18PM +, Damien Claisse wrote:
> Hi Willy,
> 
> Thanks you very much for your answer. Let's go with Tu, sounds relevant to me.
> I've also slightly updated documentation, please tell me if this is not clear
> enough.

Looks perfectly clear, and applied as-is :-)

Thanks!
Willy

[PATCH 1/1] MINOR: log: Add "Tu" timer

[Damien Claisse]

It can be sometimes useful to measure total time of a request as seen
from an end user, including TCP/TLS negotiation, server response time
and transfer time. "Tt" currently provides something close to that, but
it also takes client idle time into account, which is problematic for
keep-alive requests as idle time can be very long. "Ta" is also not
sufficient as it hides TCP/TLS negotiationtime. To improve that, introduce
a "Tu" timer, without idle time and everything else. It roughly estimates
time spent time spent from user point of view (without DNS resolution
time), assuming network latency is the same in both directions.
---
 doc/configuration.txt | 11 +++
 include/types/log.h   |  1 +
 src/log.c | 12 
 3 files changed, 24 insertions(+)

diff --git a/doc/configuration.txt b/doc/configuration.txt
index cf521483a..7dde4fc24 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -17733,6 +17733,7 @@ Please refer to the table below for currently defined 
variables :
   | H | %Tr  | Tr (response time)| numeric |
   |   | %Ts  | timestamp | numeric |
   |   | %Tt  | Tt| numeric |
+  |   | %Tu  | Tu| numeric |
   |   | %Tw  | Tw| numeric |
   |   | %U   | bytes_uploaded   (from client to server)  | numeric |
   |   | %ac  | actconn   | numeric |
@@ -17900,6 +17901,7 @@ Timings events in HTTP mode:
   : Th   Ti   TR   Tw   Tc   Tr   Td : Ti   ...
   :< Tq >:   :
   :<-- Tt -->:
+  :<---Tu--->:
 :<- Ta ->:
 
 Timings events in TCP mode:
@@ -17996,6 +17998,15 @@ Timings events in TCP mode:
 mode, "Ti", "Tq" and "Tr" have to be excluded too. Note that "Tt" can never
 be negative and that for HTTP, Tt is simply equal to (Th+Ti+Ta).
 
+  - Tu: total estimated time as seen from client, between the moment the proxy
+accepted it and the moment both ends were closed, without idle time.
+This is useful to roughly measure end-to-end time as a user would see it,
+without idle time pollution from keep-alive time between requests. This
+timer in only an estimation of time seen by user as it assumes network
+latency is the same in both directions. The exception is when the "logasap"
+option is specified. In this case, it only equals (Th+TR+Tw+Tc+Tr), and is
+prefixed with a '+' sign.
+
 These timers provide precious indications on trouble causes. Since the TCP
 protocol defines retransmit delays of 3, 6, 12... seconds, we know for sure
 that timers close to multiples of 3s are nearly always related to lost packets
diff --git a/include/types/log.h b/include/types/log.h
index c348caa1e..868c08f01 100644
--- a/include/types/log.h
+++ b/include/types/log.h
@@ -104,6 +104,7 @@ enum {
LOG_FMT_TR,
LOG_FMT_TD,
LOG_FMT_TT,
+   LOG_FMT_TU,
LOG_FMT_STATUS,
LOG_FMT_CCLIENT,
LOG_FMT_CSERVER,
diff --git a/src/log.c b/src/log.c
index 50b5080c4..7175e74ee 100644
--- a/src/log.c
+++ b/src/log.c
@@ -151,6 +151,7 @@ static const struct logformat_type logformat_keywords[] = {
{ "Td", LOG_FMT_TD, PR_MODE_TCP, LW_BYTES, NULL },   /* Td = Tt - 
(Tq + Tw + Tc + Tr) */
{ "Ts", LOG_FMT_TS, PR_MODE_TCP, LW_INIT, NULL },   /* timestamp GMT */
{ "Tt", LOG_FMT_TT, PR_MODE_TCP, LW_BYTES, NULL },   /* Tt */
+   { "Tu", LOG_FMT_TU, PR_MODE_TCP, LW_BYTES, NULL },   /* Tu = Tt -Ti 
*/
{ "Tw", LOG_FMT_TW, PR_MODE_TCP, LW_BYTES, NULL },   /* Tw */
{ "U", LOG_FMT_BYTES_UP, PR_MODE_TCP, LW_BYTES, NULL },  /* bytes from 
client to server */
{ "ac", LOG_FMT_ACTCONN, PR_MODE_TCP, LW_BYTES, NULL },  /* actconn */
@@ -2450,6 +2451,17 @@ int sess_build_logline(struct session *sess, struct 
stream *s, char *dst, size_t
last_isspace = 0;
break;
 
+   case LOG_FMT_TU:  // %Tu = total time seen by user = Tt 
- Ti
+   if (!(fe->to_log & LW_BYTES))
+   LOGCHAR('+');
+   ret = ltoa_o(logs->t_close - (logs->t_idle >= 0 
? logs->t_idle : 0),
+tmplog, dst + maxsize - tmplog);
+   if (ret == NULL)
+   goto out;
+   tmplog = ret;
+   last_isspace = 0;
+   break;
+
case LOG_FMT_STATUS: // %ST
ret = ltoa_o(txn ? txn->status : 0, tmplog, dst 
+ maxsize - tmplog);
i

[PATCH 0/1] *** Add Tu timer ***

[Damien Claisse]

Hi Willy,

Thanks you very much for your answer. Let's go with Tu, sounds relevant to me.
I've also slightly updated documentation, please tell me if this is not clear
enough.

Cheers,

Damien

Damien Claisse (1):
  MINOR: log: Add "Tu" timer

 doc/configuration.txt | 11 +++
 include/types/log.h   |  1 +
 src/log.c | 12 
 3 files changed, 24 insertions(+)

-- 
2.20.1

Re: [PATCH] remove reg-tests/checks/tcp-check-ssl.vtc on CentOS 6

[William Lallemand]

On Mon, Apr 27, 2020 at 11:37:11PM +0500, Илья Шипицин wrote:
> Hello,
> 
> new reg-test requires ALPN which is not available on CentOS 6.
> 
> Cheers,
> Ilya Shipitcin

> From f6edcbacc58ccbfe47f25fccfe6a5743fcae1122 Mon Sep 17 00:00:00 2001
> From: Ilya Shipitsin 
> Date: Mon, 27 Apr 2020 23:35:13 +0500
> Subject: [PATCH] CI: cirrus-ci: remove reg-tests/checks/tcp-check-ssl.vtc on
>  CentOS 6
> 

Thanks, merged!


-- 
William Lallemand

Check my high quality site good traffic.

[Nina Adams]

Hello Sir,

Have a look on my top quality normal & casino
sites. I hope you'll like them.
I can provide you guest post on these sites. Send me the order if you have
interest in any of the following site.
   Sincerely waiting for your reply!
General Sites DA PA Traffic
http://www.fox34.com/

59 47 16.1K
https://www.news9.com/

70 59 573K
https://www.newson6.com

72 59 696K
https://www.wrcbtv.com

77 58 211K
http://www.ktvn.com/

66 57 113K
http://www.kake.com/

69 58 224K
https://www.erienewsnow.com/

71 50 103K
http://www.wboc.com/

64 57 162K
https://www.wfxg.com/

61 49 11.4K
https://www.cbs19news.com/

53 43 61.6K
https://www.rfdtv.com/

58 56 99.3K
https://www.weny.com/

59 54 35.6K
https://www.1170kfaq.com/

50 51 4.71K
Casino Sites DA PA Traffic
www.cascadebusnews.com

48 44 4.25K
www.trans4mind.com

56 48 18.2K
www.africanexponent.com

56 42 10.7K
www.urbansplatter.com

41 39 1.06K
www.athleisuremag.com

36 32 2.71K
www.optimisticmommy.com

41 48 1.99K
www.irish-boxing.com

49 48 1.71K
www.tastefulspace.com

40 44 6.21K
www.lol-la.com

42 38 2.05K
www.seattlestravels.com

48 49 1.39K
www.theverybesttop10.com

54 49 4.49K
www.pubclub.com

48 50 12K
www.infolific.com

45 43 5.13K