clarification on documentation for sticky counter tracking duration

[Patrick Hemmer]

I'm looking to get some clarification around the documentation for the
duration of sticky counter tracking. There are 2 specific points I'm a
little confused on.

1. Under the documentation for `tcp-request content`, it says:
> In case of HTTP keep-alive with the client, all tcp-request content
rules are evaluated again, so haproxy keeps a record of what sticky
counters were assigned by a "tcp-request connection" versus a
"tcp-request content" rule, and flushes all the content-related ones
after processing an HTTP request

This sounds like if I were to use `tcp-request content track-sc0 src`,
that after the http request is finished processing, the sticky counter
for the source IP will be stop being tracked. However this does not
appear to be the case as when I log sc0_conn_cur, it is counting clients
that are sitting idle between http requests (using http keep-alive).

2. Under the documentation for `http-request track-sc0`, it says:
> Once a "track-sc*" rule is executed, the key is looked up in the table
and if it is not found, an entry is allocated for it. Then a pointer to
that entry is kept during all the session's life

This sounds like the sticky counter is tracked for the duration of the
http session: "entry is kept during all the **session's** life". However
this does not seem to be the case as when I log sc0_conn_cur, clients
that are sitting idle between http requests are not counted.


Am I interpreting the documentation incorrectly, or is the documentation
incorrect?


-Patrick

[PATCH] bis contrib mod security

[David CARLIER]

Hi apologies the patch sent yesterday was not the correct one.

Kind regards.


0001-BUG-MINOR-contrib-modsecurity-BSD-build-fix.patch
Description: Binary data

Question regarding HAProxy config

[JOHN PEIRCE]

Hi,

I am configuring a third party application and am trying to use HAProxy to
server as a single point of access to my server pool.

Users connect to the web server which in turn is configured to talk to a
single dedicated app server.

What I would like is for HAProxy to check the app servers, and only if they
are up/listening, to then allow the connection to the web server.

So for an example, if app2 goes down, I want HAProxy to remove app2/web2
from the pool.

Is this possible?

[image: Inline image 1]

Thanks,
-j
-- 

John Peirce
Systems Administrator | *PERRY ELLIS*
*INTERNATIONAL*T (305) 873-1298 | F  (786) 221-8298 | E j...@pery.com

[PATCH v3] MINOR: ssl: add prefer-client-ciphers

[Lukas Tribus]

Currently we unconditionally set SSL_OP_CIPHER_SERVER_PREFERENCE [1],
which may not always be a good thing.

The benefit of server side cipher prioritization may not apply to all
cases out there, and it appears that the various SSL libs are going away
from this recommendation ([2], [3]), as insecure ciphers suites are
properly blacklisted/removed and honoring the client's preference is
more likely to improve user experience  (for example using SW-friendly
ciphers on devices without HW AES support).

This is especially true for TLSv1.3, which will restrict the cipher
suites to just AES-GCM and Chacha20/Poly1305.

Apache [4], nginx [5] and others give admins full flexibility, we should
as well.

The initial proposal to change the current default and add a
"prefer-server-ciphers" option (as implemented in e566ecb) has been
declined due to the possible security impact.

This patch implements prefer-client-ciphers without changing the defaults.

[1] https://www.openssl.org/docs/man1.0.2/ssl/SSL_CTX_set_options.html
[2] https://github.com/openssl/openssl/issues/541
[3] https://github.com/libressl-portable/portable/issues/66
[4] https://httpd.apache.org/docs/2.0/en/mod/mod_ssl.html#sslhonorcipherorder
[5] 
https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_prefer_server_ciphers
---
V3:
Fix clearing SSL_OP_CIPHER_SERVER_PREFERENCE when force-tlsXY methods
are used

V2:
patch doesn't change defaults and implements prefer-client-ciphers
instead of "prefer-server-ciphers".

---
 doc/configuration.txt|  5 +
 include/types/listener.h |  1 +
 src/ssl_sock.c   | 12 
 3 files changed, 18 insertions(+)

diff --git a/doc/configuration.txt b/doc/configuration.txt
index 06a1a2a..fea06cb 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -10592,6 +10592,11 @@ npn 
   enabled (check with haproxy -vv). Note that the NPN extension has been
   replaced with the ALPN extension (see the "alpn" keyword).
 
+prefer-client-ciphers
+  Use the client's preference when selecting the cipher suite, by default
+  the server's preference is enforced. This option is also available on
+  global statement "ssl-default-bind-options".
+
 process [ all | odd | even | [-] ]
   This restricts the list of processes on which this listener is allowed to
   run. It does not enforce any process but eliminates those which do not match.
diff --git a/include/types/listener.h b/include/types/listener.h
index 2b8f5fe..8aae395 100644
--- a/include/types/listener.h
+++ b/include/types/listener.h
@@ -114,6 +114,7 @@ enum li_state {
 #define BC_SSL_O_USE_TLSV12 0x0080 /* force TLSv12 */
 /* 0x00F0 reserved for 'force' protocol version options */
 #define BC_SSL_O_NO_TLS_TICKETS 0x0100 /* disable session resumption tickets */
+#define BC_SSL_O_PREF_CLIE_CIPH 0x0200  /* prefer client ciphers */
 #endif
 
 /* ssl "bind" settings */
diff --git a/src/ssl_sock.c b/src/ssl_sock.c
index 48ad1b2..acb7d28 100644
--- a/src/ssl_sock.c
+++ b/src/ssl_sock.c
@@ -3238,6 +3238,8 @@ ssl_sock_initial_ctx(struct bind_conf *bind_conf)
}
if (conf_ssl_options & BC_SSL_O_NO_TLS_TICKETS)
ssloptions |= SSL_OP_NO_TICKET;
+   if (conf_ssl_options & BC_SSL_O_PREF_CLIE_CIPH)
+   ssloptions &= ~SSL_OP_CIPHER_SERVER_PREFERENCE;
SSL_CTX_set_options(ctx, ssloptions);
SSL_CTX_set_mode(ctx, sslmode);
if (global_ssl.life_time)
@@ -6327,6 +6329,13 @@ static int bind_parse_ssl(char **args, int cur_arg, 
struct proxy *px, struct bin
return 0;
 }
 
+/* parse the "prefer-client-ciphers" bind keyword */
+static int bind_parse_pcc(char **args, int cur_arg, struct proxy *px, struct 
bind_conf *conf, char **err)
+{
+conf->ssl_options |= BC_SSL_O_PREF_CLIE_CIPH;
+return 0;
+}
+
 /* parse the "generate-certificates" bind keyword */
 static int bind_parse_generate_certs(char **args, int cur_arg, struct proxy 
*px, struct bind_conf *conf, char **err)
 {
@@ -6870,6 +6879,8 @@ static int ssl_parse_default_bind_options(char **args, 
int section_type, struct
}
else if (!strcmp(args[i], "no-tls-tickets"))
global_ssl.listen_default_ssloptions |= 
BC_SSL_O_NO_TLS_TICKETS;
+   else if (!strcmp(args[i], "prefer-client-ciphers"))
+   global_ssl.listen_default_ssloptions |= 
BC_SSL_O_PREF_CLIE_CIPH;
else {
memprintf(err, "unknown option '%s' on global statement 
'%s'.", args[i], args[0]);
return -1;
@@ -7505,6 +7516,7 @@ static struct bind_kw_list bind_kws = { "SSL", { }, {
{ "tls-ticket-keys",   bind_parse_tls_ticket_keys, 1 }, /* set file 
to load TLS ticket keys from */
{ "verify",bind_parse_verify,  1 }, /* set SSL 
verify method */
{ "npn",   bind_parse_npn, 1 }, /* set NPN 
supported protocols */
+   { "prefer-client-ciphers", 

AW: [PATCH v2] MINOR: ssl: add prefer-client-ciphers

[Lukas Tribus]

> I'm just waiting for Emeric's approval to merge it.

Don't commit yet, there's a little bug in it, I will send
a v3 shortly.

Sorry about that,
lukas

Re: GeoIP Best Practices

[Arnaud B.]

After a quicksearch in Haproxy's doc I've found a way to do what you
suggested :

http-request set-header X-City-Map
%[src,map_ip(/etc/haproxy/geocity_map.lst)]
http-request set-header X-Country-Map
%[src,map_ip(/etc/haproxy/geocountry_map.lst)]
http-request set-header X-City-Name %[hdr(X-City-Map),word(1,' ')]
http-request set-header X-Geoname-City-Id 
%[hdr(X-City-Map),word(2,' ')]
http-request set-header X-Latitude  %[hdr(X-City-Map),word(3,' ')]
http-request set-header X-Longitude  %[hdr(X-City-Map),word(4,' ')]
http-request set-header X-Accuracy-Radius 
%[hdr(X-City-Map),word(5,' ')]
http-request set-header X-Country-Code %[hdr(X-Country-Map),word(1,' ')]
http-request set-header X-Country-Code2
%[hdr(X-Country-Map),word(1,' ')]
http-request set-header X-Country-Id %[hdr(X-Country-Map),word(2,' ')]
http-request del-header X-City-Map
http-request del-header X-Country-Map

My city mapfile is around 800Mib and HAproxy uses 2 times less RAM and
CPU after this modification, this setup is now extensible at will.


Thank's Willy for the head's up, I hope it'll be useful to somebody else :-)


04/05/2017 à 07:30, Willy Tarreau a écrit :
> Hi Arnaud,
>
> On Wed, May 03, 2017 at 01:54:13PM +0200, Arnaud B. wrote:
>> Hi there,
>>
>> I'm currently wondering, based on
>> https://www.haproxy.com/blog/use-geoip-database-within-haproxy/ and
>> related notes, is there a more convenient way now ? I've created
>> mapfiles for latitude, longitude, accuracy radius, country code, cities
>> names etc. 
>>
>> Right now, my process is based on Maxmind's databases scraping and
>> generating mapfiles for each added header.
>>
>> Now my HAProxy is around 10GB ram and takes about 70secs to reload.
> Why does it take that long ? That seems totally abnormal. Do you use
> your maps at many places, possibly causing them from being loaded
> multiple times maybe ? In theory even with this they should be merged.
>
> Now regarding the size, do you think your maps could be aggregated
> so that you have all fields at once on a line ? If so you could set
> a variable based on a single lookup of the source address in the
> map, then use the "word" converter to split it depending on the
> field you're interested in. The risk is to end up with an even larger
> map but I think that most entries can be merged.
>
> Willy
>
>

Re: [RFC][PATCHES] seamless reload

[Olivier Houchard]

On Thu, May 04, 2017 at 10:03:07AM +, Pierre Cheynier wrote:
> Hi Olivier,
> 
> Many thanks for that ! As you know, we are very interested on this topic.
> We'll test your patches soon for sure.
> 
> Pierre

Hi Pierre :)

Thanks ! I'm very interested in knowing how well it works for you.
Maybe we can talk about that around a beer sometime.

Olivier

Re: DNS and bind statement

[Stephan Mueller]

> > > Well, you're missing a port number here and when I do this I get
> > > the same error for both : "missing port number". However when I
> > > add the port, both work for me. Eg:
> > >
> > >   frontend foo
> > > bind 1wt.eu:8002
> > >
> > > Are you sure you're not facing a libc-specific issue (check your
> > > nsswitch.conf to ensure that "dns" is not disabled for
> > > example) ?  
> >
> > I rechecked and it worked as expected. It probably was an resolver
> > issue.
> >
> >  Thank you
> >
> >  
> Hi Stephan,
> 
> I'm interested by this use case.
> When do you need a hostname in the bind line?
> Do you think it would make sense to resolve it at run time, if, for
> example, the IP pointed by hostname changes frequently?
> 
> Baptiste

Well, I use it to keep my configuration clean, from time to time I
change an IP address. So in this case I don't have to touch the
configuration. Moreover it makes the config much more readable.

Of course, haproxy needs a reload, but that is no Problem for me.
I have no need to reflect DNS changes at runtime - yet :)

RE: [RFC][PATCHES] seamless reload

[Pierre Cheynier]

Hi Olivier,

Many thanks for that ! As you know, we are very interested on this topic.
We'll test your patches soon for sure.

Pierre

Re: DNS and bind statement

[Baptiste]

>
> > Well, you're missing a port number here and when I do this I get the
> > same error for both : "missing port number". However when I add the
> > port, both work for me. Eg:
> >
> >   frontend foo
> > bind 1wt.eu:8002
> >
> > Are you sure you're not facing a libc-specific issue (check your
> > nsswitch.conf to ensure that "dns" is not disabled for example) ?
>
> I rechecked and it worked as expected. It probably was an resolver
> issue.
>
>  Thank you
>
>
Hi Stephan,

I'm interested by this use case.
When do you need a hostname in the bind line?
Do you think it would make sense to resolve it at run time, if, for
example, the IP pointed by hostname changes frequently?

Baptiste

Re: GeoIP Best Practices

[Arnaud B.]

Hi Willy,

Thank's for the quick reply and helpful answers.

I have a multiproc HAProxy, so my first guess is that each proc has to
have the mapfiles in memory.

Each mapfile has a CIDR IP block as key, and I can merge them based on
this indeed. I'll try to use the word converter to apply the right
headers based on a merged file and will get back to the mailinglist to
let you know how it went.



Le 04/05/2017 à 07:30, Willy Tarreau a écrit :
> Hi Arnaud,
>
> On Wed, May 03, 2017 at 01:54:13PM +0200, Arnaud B. wrote:
>> Hi there,
>>
>> I'm currently wondering, based on
>> https://www.haproxy.com/blog/use-geoip-database-within-haproxy/ and
>> related notes, is there a more convenient way now ? I've created
>> mapfiles for latitude, longitude, accuracy radius, country code, cities
>> names etc. 
>>
>> Right now, my process is based on Maxmind's databases scraping and
>> generating mapfiles for each added header.
>>
>> Now my HAProxy is around 10GB ram and takes about 70secs to reload.
> Why does it take that long ? That seems totally abnormal. Do you use
> your maps at many places, possibly causing them from being loaded
> multiple times maybe ? In theory even with this they should be merged.
>
> Now regarding the size, do you think your maps could be aggregated
> so that you have all fields at once on a line ? If so you could set
> a variable based on a single lookup of the source address in the
> map, then use the "word" converter to split it depending on the
> field you're interested in. The risk is to end up with an even larger
> map but I think that most entries can be merged.
>
> Willy
>

AW: [PATCH v2] MINOR: ssl: add prefer-client-ciphers

[Lukas Tribus]

>> SSL_OP_CIPHER_SERVER_PREFERENCE is not evil. But yeah - we do want to have
>> maximal flexibility in every case.
>
> Does this mean that this should also be backported to 1.7 in your opinion ?
> Maybe even older versions ?

Yes, at this point (since the v2 patch doesn't change the default behavior) I 
believe
it makes sense to backport this to 1.7. I don't have a strong opinion about 
1.6, whatever
you decide works fine.



> I'm just waiting for Emeric's approval to merge it.

Great, thanks.

Lukas

AW: [PATCH v2] MINOR: ssl: add prefer-client-ciphers

[Lukas Tribus]

> Can client "override" servers ssl-default-server-ciphers/bind ciphers(
> or is the cipher suite selected from ssl-default-server-ciphers/ciphers)?

Both the server and the client have a list of supported cipher suites, ordered
by preference. With SSL_OP_CIPHER_SERVER_PREFERENCE enabled, the
server ignores the client preferences using its own. Otherwise the clients
preference is honored (that's what prefer-client-ciphers does).

On the backend (ssl-default-server-ciphers), where haproxy only acts as a
TLS client, the behavior depends completely on the server, we can provide
our list including preferences (ssl-default-server-ciphers), but the server
decides whether our preferences is respected or not.

That's why this option is only relevant for bind, not for servers.

A client can always enforce a specific cipher suite though, by specifying
only a single cipher in its list. If the server support and enables that cipher
suite, the connection succeeds, otherwise it fails.


Lukas

Re: ModSecurity: First integration patches

[Aleksandar Lazic]

Am Thu, 4 May 2017 08:57:41 +0200
schrieb Baptiste :

> On Thu, May 4, 2017 at 8:50 AM, Aleksandar Lazic 
> wrote:
> 
> > Hi.
> >
> > awsome timing because nginx just released his WAF also ;-).
> >
> > https://www.nginx.com/blog/modsecurity-waf-released/
>
> Well, you can't compare a proprietary product with an open source one!
> And here, we see the benefits of the community behind such product.

Well I see the benefit in the same requirements.
Both solutin need a independent modsecurity => libmodsecurity.

> Baptiste
> 
Aleks

Re: ModSecurity: First integration patches

[Baptiste]

On Thu, May 4, 2017 at 8:50 AM, Aleksandar Lazic  wrote:

> Hi.
>
> awsome timing because nginx just released his WAF also ;-).
>
> https://www.nginx.com/blog/modsecurity-waf-released/
>
>
Well, you can't compare a proprietary product with an open source one!
And here, we see the benefits of the community behind such product.

Baptiste

Re: Failed to compile haproxy with lua on Solaris 10

[Benoît GARNIER]

Le 04/05/2017 à 07:14, Willy Tarreau a écrit :
> Hi,
>
> On Wed, May 03, 2017 at 06:17:40AM +, Akhnin Nikita wrote:
>> Greetings!
>>
>> I have a problem with compiling haproxy with lua support against Solaris 10.
>> I'm using gcc v4.9 and making it with command (don't pay attention to the 
>> odd paths to the gcc and libraries):
>> gmake CC=/pub/site/opt/bin/gcc TARGET=solaris USE_OPENSSL=1 
>> SSL_INC=/pub/site/opt/include/openssl SSL_LIB=/pub/site/opt/lib USE_PCRE=1 
>> PCREDIR=/pub/site/opt PREFIX=/pub/site/opt USE_PCRE_JIT=1 USE_STATIC_PCRE=1 
>> USE_ZLIB=1 USE_LUA=1 ADDLIB="-R/pub/site/opt/lib" LUA_LIB=/pub/site/opt/lib 
>> LUA_INC=/pub/site/opt/include
>>
>> The error message is:
>>
>> /pub/site/opt/bin/gcc -Iinclude -Iebtree -Wall  -O2 -g -fno-strict-aliasing 
>> -Wdeclaration-after-statement -fomit-frame-pointer -DFD_SETSIZE=65536 
>> -D_REENTRANT  -DTPROXY -DCONFIG_HAP_CRYPT -DNEED_CRYPT_H 
>> -DUSE_GETADDRINFO -DUSE_ZLIB  -DENABLE_POLL -DUSE_OPENSSL 
>> -I/pub/site/opt/include/openssl -DUSE_LUA  -DUSE_PCRE 
>> -I/pub/site/opt/include -DUSE_PCRE_JIT  -DCONFIG_HAPROXY_VERSION=\"1.7.4\" 
>> -DCONFIG_HAPROXY_DATE=\"2017/03/27\" -c -o src/hlua_fcn.o src/hlua_fcn.c
>> src/hlua_fcn.c: In function 'hlua_parse_date':
>> src/hlua_fcn.c:290:2: warning: implicit declaration of function 'timegm' 
>> [-Wimplicit-function-declaration]
>>   time = timegm();
>>   ^
>>
>> /pub/site/opt/bin/gcc  -g -o haproxy src/haproxy.o src/base64.o 
>> src/protocol.o src/uri_auth.o src/standard.o src/buffer.o src/log.o 
>> src/task.o src/chunk.o src/channel.o src/listener.o src/lru.o src/xxhash.o 
>> src/time.o src/fd.o src/pipe.o src/regex.o src/cfgparse.o src/server.o 
>> src/checks.o src/queue.o src/frontend.o src/proxy.o src/peers.o src/arg.o 
>> src/stick_table.o src/proto_uxst.o src/connection.o src/proto_http.o 
>> src/raw_sock.o src/backend.o src/tcp_rules.o src/lb_chash.o src/lb_fwlc.o 
>> src/lb_fwrr.o src/lb_map.o src/lb_fas.o src/stream_interface.o src/stats.o 
>> src/proto_tcp.o src/applet.o src/session.o src/stream.o src/hdr_idx.o 
>> src/ev_select.o src/signal.o src/acl.o src/sample.o src/memory.o 
>> src/freq_ctr.o src/auth.o src/proto_udp.o src/compression.o src/payload.o 
>> src/hash.o src/pattern.o src/map.o src/namespace.o src/mailers.o src/dns.o 
>> src/vars.o src/filters.o src/flt_http_comp.o src/flt_trace.o src/flt_spoe.o 
>> src/cli.o src/ev_poll.o src/ssl_sock.o src/shctx.o src/hlua.o src/hlua_fcn.o 
>> ebtree/ebtree.o ebtree/eb32tree.o ebtree/eb64tree.o ebtree/ebmbtree.o 
>> ebtree/ebsttree.o ebtree/ebimtree.o ebtree/ebistree.o -lnsl -lsocket  
>> -lcrypt  -lz -L/pub/site/opt/lib -lssl -lcrypto -Wl,--export-dynamic  -llua 
>> -lm -L/pub/site/opt/lib -Wl,-Bstatic -lpcreposix -lpcre -Wl,-Bdynamic 
>> -R/pub/site/opt/lib
>> src/hlua_fcn.o: In function `hlua_parse_date':
>> /pub/home/appadm/temp/haproxy-1.7.4/src/hlua_fcn.c:290: undefined reference 
>> to `timegm'
>> collect2: error: ld returned 1 exit status
>> gmake: *** [haproxy] Error 1
>>
>> According to 
>> http://stackoverflow.com/questions/40904201/alternative-to-timegm-on-solaris,
>>  there is no timegm function in Solaris, so I can't solve this without power 
>> of haproxy developers.
>>
>> Is there any way to make this code compatible with Solaris?
> I had never heard about this function. Could you try with mktime() instead ?
> It takes the same prototype. If anybody knows what the difference is between
> the two, I'll appreciate any enlightment on this. The man pages are not clear
> enough for me on the subject.

There are two ways to build a struct tm to break down the time in hour,
minute, second and so on: calling localtime, which takes current
timezone into account, or calling gmtime to build an UTC time.

The reverse operation is done by calling mktime (which will give the
original time_t only if the current timezone is the same as when
localtime was called, since the timezone is not recorded in struct tm)
in the former case or timegm in the latter.

If you do the following operation : time_t => localtime() => struct tm
=> timegm() => time_t, your result will be shift by the timezone time
offset (but without any DST applied).

Technically, if you live in Great Britain, the operation will succeed
during winter (but will offset the result by 1 hour during summer, since
DST is applied here).


Benoît



---
L'absence de virus dans ce courrier électronique a été vérifiée par le logiciel 
antivirus Avast.
https://www.avast.com/antivirus

Re: ModSecurity: First integration patches

[Aleksandar Lazic]

Hi.

awsome timing because nginx just released his WAF also ;-).

https://www.nginx.com/blog/modsecurity-waf-released/

Now we only need to give Willy the time and chance to work on
HTTP/2 to have this feature also in haproxy ;-)

Regards
Aleks

Am Thu, 27 Apr 2017 23:10:15 +0200
schrieb Thierry Fournier :

> > On 27 Apr 2017, at 18:53, Aleksandar Lazic 
> > wrote:
> > 
> > Hi Willy.
> > 
> > Am 27-04-2017 12:05, schrieb Willy Tarreau:  
> >> Hi Thierry,
> >> On Thu, Apr 20, 2017 at 03:05:35PM +0200, Thierry Fournier wrote:  
> >>> Hi,
> >>> After a quick private brainstorm, Willy propose to me a new
> >>> binary encoding for the headers. It is useless to give the
> >>> numbers of headers contained in the block, so the end of headers
> >>> is marked by an empty header (header name and value have a length
> >>> of 0). The new patches in attachment (first patch is 0002).  
> >> OK I finally merged your patchset, but I fixed a few typos and a
> >> bit of confusing wording in the doc.  
> > 
> > I'm not Thierry but I say, thanks ;-).  
> 
> 
> I’m Thierry and I say thanks too ;-)
> 
> 
> > This make my docker setup easier.
> > 
> > I will write a blog entry about this, I will share it here.  
> 
> 
> great, share the link !
> 
> 
> >   
> >> Thanks,
> >> Willy  
> > 
> > Very best regards
> > Aleks  
>