Re: Compilation problem: haproxy 1.6.5 (latest) on Solaris 11

[Vincent Bernat]

 ❦ 18 mai 2016 22:56 +0200, Pavlos Parissis  :

>> Also, where is the bugtracker for haproxy? I can file a report if you
>> want to save time.
>
> As far as I know there isn't any bugtracker. Posting problems in this
> ML is enough to kick the investigation. So far this model works quite
> well

Yes, Willy will notice the patch at some point and maybe merge it if
he's OK with it.
-- 
Habit is habit, and not to be flung out of the window by any man, but coaxed
down-stairs a step at a time.
-- Mark Twain, "Pudd'nhead Wilson's Calendar

Re: [PATCH] BUG/MEDIUM: init: don't use environment locale

[Willy Tarreau]

On Wed, May 18, 2016 at 11:13:38PM +0200, Maxime de Roucy wrote:
> This patch remove setlocale from the main function.
(...)

Thank you Maxime, I've merged it. I've amended your commit message
to mention the ID of the commit which introduced it in order to
avoid future confusion from people who fear a change of behaviour.

Thanks!
Willy

Re: [PATCH] BUG/MEDIUM: init: don't use environment locale

[Maciej Katafiasz]

On 18 May 2016 at 16:18, Cyril Bonté  wrote:
> Le 19/05/2016 00:34, Maciej Katafiasz a écrit :
>> While potentially
>> confusing, forcing it to C is also confusing and prevents people from
>> actually exploiting locale should they want to, and traditionally the
>> Unix approach was to let the administrator get the locale right, so
>> it's also more consistent with how other software does it.
>
>
> Which ones ? As previously said, nor apache httpd and nginx do that with
> their respective "include" directive, at least. Why would we want to make it
> differently ?

Postgresql for example, and many other commonly used pieces of unix
code. Like shell, as was originally pointed out.

>> It's
>> documented that way too, so your patch constitutes a design change,
>> which I feel should be argued for with a bit more rationale.
>
>
> It only reverts to the original design.

Right, I got confused about the context of the change, and missed the
fact it was within the config dir patch only. Still, both approaches
have their downsides, as also mentioned. That said, not touching
existing configs is probably the better side to err on, so I withdraw
my objections.

I'd spend a bit more text on explaining the implications of collation
in C locale vs user locale though, change "LC_COLLATE=C" in the docs
to "C locale" because the code does not actually consult LC_COLLATE,
clarify what "lexical order" means in the context of multiple
directories (ie. whether all files found are sorted together before
including, or only within the containing directory, with the order of
directories passed determining the order of inclusion), and that
directories are *not* scanned recursively (which is my understanding
of the intended design).

Cheers,
Maciej

Re: [PATCH] BUG/MEDIUM: init: don't use environment locale

[Cyril Bonté]

Le 19/05/2016 00:34, Maciej Katafiasz a écrit :

On 18 May 2016 at 14:13, Maxime de Roucy  wrote:


This patch remove setlocale from the main function.

Some regex may have different behaviours depending on the
locale. Some LUA scripts may change their behaviour too
(http://lua-users.org/wiki/LuaLocales).


I'm not convinced that is a good change. It should probably be left up
to the environment to get the locale right.


On the contrary, I'm more convinced its a good one.


While potentially
confusing, forcing it to C is also confusing and prevents people from
actually exploiting locale should they want to, and traditionally the
Unix approach was to let the administrator get the locale right, so
it's also more consistent with how other software does it.


Which ones ? As previously said, nor apache httpd and nginx do that with 
their respective "include" directive, at least. Why would we want to 
make it differently ?



It's
documented that way too, so your patch constitutes a design change,
which I feel should be argued for with a bit more rationale.


It only reverts to the original design.


--
Cyril Bonté

Re: [PATCH] BUG/MEDIUM: init: don't use environment locale

[Maciej Katafiasz]

On 18 May 2016 at 14:13, Maxime de Roucy  wrote:
>
> This patch remove setlocale from the main function.
>
> Some regex may have different behaviours depending on the
> locale. Some LUA scripts may change their behaviour too
> (http://lua-users.org/wiki/LuaLocales).

I'm not convinced that is a good change. It should probably be left up
to the environment to get the locale right. While potentially
confusing, forcing it to C is also confusing and prevents people from
actually exploiting locale should they want to, and traditionally the
Unix approach was to let the administrator get the locale right, so
it's also more consistent with how other software does it. It's
documented that way too, so your patch constitutes a design change,
which I feel should be argued for with a bit more rationale.

[PATCH] BUG/MEDIUM: init: don't use environment locale

[Maxime de Roucy]

This patch remove setlocale from the main function.

Some regex may have different behaviours depending on the
locale. Some LUA scripts may change their behaviour too
(http://lua-users.org/wiki/LuaLocales).

Without this patch (haproxy is using setlocale) :

$ cat locale.cfg
defaults
  mode http

frontend test
  bind :9000
  mode http
  use_backend testbk if { hdr_reg(X-Test) ^\w+$ }

backend testbk
  mode http
  server s 127.0.0.1:80

$ LANG=fr_FR.UTF-8 ./haproxy -f locale.cfg
$ curl -i -H "X-Test: échec" localhost:9000
HTTP/1.1 200 OK
...

$ LANG=C ./haproxy -f locale.cfg
$ curl -i -H "X-Test: échec" localhost:9000
HTTP/1.0 503 Service Unavailable
...
---
 doc/haproxy.1  | 4 ++--
 doc/management.txt | 2 +-
 src/haproxy.c  | 4 
 3 files changed, 3 insertions(+), 7 deletions(-)

diff --git a/doc/haproxy.1 b/doc/haproxy.1
index cc9c702..73c2aee 100644
--- a/doc/haproxy.1
+++ b/doc/haproxy.1
@@ -35,8 +35,8 @@ instances without risking the system's stability.
 .TP
 \fB\-f \fP
 Specify configuration file or directory path. If the argument is a directory
-the files (and only files) it containes are added in lexical order (respecting
-LC_COLLATE) ; only non hidden files with ".cfg" extension are added.
+the files (and only files) it containes are added in lexical order (using
+LC_COLLATE=C) ; only non hidden files with ".cfg" extension are added.
 
 .TP
 \fB\-L \fP
diff --git a/doc/management.txt b/doc/management.txt
index 4f0af10..119c3fa 100644
--- a/doc/management.txt
+++ b/doc/management.txt
@@ -136,7 +136,7 @@ list of options is :
 
   -f  : adds  to the list of configuration files to be
 loaded. If  is a directory, all the files (and only files) it
-containes are added in lexical order (respecting LC_COLLATE) to the list of
+containes are added in lexical order (using LC_COLLATE=C) to the list of
 configuration files to be loaded ; only files with ".cfg" extension are
 added, only non hidden files (not prefixed with ".") are added.
 Configuration files are loaded and processed in their declaration order.
diff --git a/src/haproxy.c b/src/haproxy.c
index 582ad82..c6d1505 100644
--- a/src/haproxy.c
+++ b/src/haproxy.c
@@ -32,7 +32,6 @@
 #include 
 #include 
 #include 
-#include 
 #include 
 #include 
 #include 
@@ -1752,9 +1751,6 @@ int main(int argc, char **argv)
char errmsg[100];
int pidfd = -1;
 
-   /* get the locale from the environment variables */
-   setlocale(LC_ALL, "");
-
init(argc, argv);
signal_register_fct(SIGQUIT, dump, SIGQUIT);
signal_register_fct(SIGUSR1, sig_soft_stop, SIGUSR1);
-- 
2.8.2

[PATCH] BUG/MEDIUM: init: don't use environment locale

[Maxime de Roucy]

This patch remove setlocale from the main function.

Some regex may have different behaviours depending on the
locale. Some LUA scripts may change their behaviour too
(http://lua-users.org/wiki/LuaLocales).

Without this patch (haproxy is using setlocale) :

$ cat locale.cfg
defaults
  mode http

frontend test
  bind :9000
  mode http
  use_backend testbk if { hdr_reg(X-Test) ^\w+$ }

backend testbk
  mode http
  server s 127.0.0.1:80

$ LANG=fr_FR.UTF-8 ./haproxy -f locale.cfg
$ curl -i -H "X-Test: échec" localhost:9000
HTTP/1.1 200 OK
...

$ LANG=C ./haproxy -f locale.cfg
$ curl -i -H "X-Test: échec" localhost:9000
HTTP/1.0 503 Service Unavailable
...
---
 doc/haproxy.1  | 4 ++--
 doc/management.txt | 2 +-
 src/haproxy.c  | 4 
 3 files changed, 3 insertions(+), 7 deletions(-)

diff --git a/doc/haproxy.1 b/doc/haproxy.1
index cc9c702..73c2aee 100644
--- a/doc/haproxy.1
+++ b/doc/haproxy.1
@@ -35,8 +35,8 @@ instances without risking the system's stability.
 .TP
 \fB\-f \fP
 Specify configuration file or directory path. If the argument is a directory
-the files (and only files) it containes are added in lexical order (respecting
-LC_COLLATE) ; only non hidden files with ".cfg" extension are added.
+the files (and only files) it containes are added in lexical order (using
+LC_COLLATE=C) ; only non hidden files with ".cfg" extension are added.
 
 .TP
 \fB\-L \fP
diff --git a/doc/management.txt b/doc/management.txt
index 4f0af10..119c3fa 100644
--- a/doc/management.txt
+++ b/doc/management.txt
@@ -136,7 +136,7 @@ list of options is :
 
   -f  : adds  to the list of configuration files to be
 loaded. If  is a directory, all the files (and only files) it
-containes are added in lexical order (respecting LC_COLLATE) to the list of
+containes are added in lexical order (using LC_COLLATE=C) to the list of
 configuration files to be loaded ; only files with ".cfg" extension are
 added, only non hidden files (not prefixed with ".") are added.
 Configuration files are loaded and processed in their declaration order.
diff --git a/src/haproxy.c b/src/haproxy.c
index 582ad82..c6d1505 100644
--- a/src/haproxy.c
+++ b/src/haproxy.c
@@ -32,7 +32,6 @@
 #include 
 #include 
 #include 
-#include 
 #include 
 #include 
 #include 
@@ -1752,9 +1751,6 @@ int main(int argc, char **argv)
char errmsg[100];
int pidfd = -1;
 
-   /* get the locale from the environment variables */
-   setlocale(LC_ALL, "");
-
init(argc, argv);
signal_register_fct(SIGQUIT, dump, SIGQUIT);
signal_register_fct(SIGUSR1, sig_soft_stop, SIGUSR1);
-- 
2.8.2

Re: [PATCH] BUG/MAJOR: fix listening IP address storage for frontends

[Cyril Bonté]

Le 18/05/2016 22:52, Arthur Țițeică a écrit :

În ziua de miercuri, 18 mai 2016, la 22:38:45 EEST, Cyril Bonté a scris:

It looks like you didn't recompile with USE_OPENSSL=1
haproxy -vv should give some hints.


Indeed, sorry about that. And error on my build script.

I fixed that, haproxy starts successfully but the issue remains, it still
listens on the wildcard IPv4 addresses.


Are you sure to run the binary with the patch applied ?

From my quick test, Vincent's patch is working well with gcc 6 (and 
that's already what I saw during my diagnostics about memcpy).




--
Cyril Bonté

Re: Compilation problem: haproxy 1.6.5 (latest) on Solaris 11

[Pavlos Parissis]

On 18/05/2016 10:42 μμ, Jonathan Fisher wrote:
> Also, where is the bugtracker for haproxy? I can file a report if you want to 
> save time.
> 

As far as I know there isn't any bugtracker. Posting problems in this ML is 
enough to kick the
investigation. So far this model works quite well

Cheers,
Pavlos



signature.asc
Description: OpenPGP digital signature

Re: [PATCH] BUG/MAJOR: fix listening IP address storage for frontends

[Arthur Țițeică]

În ziua de miercuri, 18 mai 2016, la 22:38:45 EEST, Cyril Bonté a scris:
> It looks like you didn't recompile with USE_OPENSSL=1
> haproxy -vv should give some hints.

Indeed, sorry about that. And error on my build script.

I fixed that, haproxy starts successfully but the issue remains, it still 
listens on the wildcard IPv4 addresses.

Re: Compilation problem: haproxy 1.6.5 (latest) on Solaris 11

[Jonathan Fisher]

Also, where is the bugtracker for haproxy? I can file a report if you want
to save time.

On Wed, May 18, 2016 at 1:52 PM, Jonathan Fisher 
wrote:

> Nice here's the complication output:
>
>
> http://pastebin.com/iS2JKXED
>
> Now I just have to figure out how to add openssl, zlib, and libpcre which
> don't seem to be available on Oracle Solaris.
>
> On Wed, May 18, 2016 at 1:40 PM, Vincent Bernat  wrote:
>
>> You can try this patch to check if it works.
>>
>>
>>
>> --
>> Let the data structure the program.
>> - The Elements of Programming Style (Kernighan & Plauger)
>>
>>  ――― Original Message ―――
>>  From: Jonathan Fisher 
>>  Sent: 18 mai 2016 12:56 -0400
>>  Subject: Re: Compilation problem: haproxy 1.6.5 (latest) on Solaris 11
>>  To: Vincent Bernat
>>  Cc: haproxy@formilux.org
>>
>>
>> > Oh man, I wish I was smart enough to understand what's happening
>> > there... what do I need to replace? (And for my own learning/benefit,
>> > can you explain what's happening?)
>> >
>> > Thank you!!!
>> >
>> > On Wed, May 18, 2016 at 12:07 PM, Vincent Bernat 
>> > wrote:
>> >
>> > Hey!
>> >
>> > Since there is some discrepancy in the definition of ntohll among
>> > platforms, I define this macro to shadow any existing definition:
>> >
>> > #ifndef ntohll
>> > # define ntohll(x) \
>> > (((u_int64_t)(ntohl((int)(((x) << 32) >> 32))) << 32) | \
>> > (unsigned int)ntohl(((int)((x) >> 32
>> > #endif
>> >
>> > You can try yourself. Or if you don't know what to replace, I can
>> > provide a patch.
>> > --
>> > Don't compare floating point numbers just for equality.
>> > - The Elements of Programming Style (Kernighan & Plauger)
>> >
>> > ――― Original Message ―――
>> > From: Jonathan Fisher 
>> > Sent: 18 mai 2016 10:22 -0400
>> > Subject: Compilation problem: haproxy 1.6.5 (latest) on Solaris 11
>> > To: haproxy@formilux.org
>> >
>> >
>> >
>> > > We can get 1.5 to compile quite nicely on Solaris 11 :) 1.6 has
>> > a few
>> > > nice features though we'd like to have, if it all possible. I
>> > have a
>> > > VMDK created if the developers want to try it themselves in a
>> > virtual
>> > > machine.
>> > >
>> > > Here is the output of the compiler, which is GCC-48:
>> > >
>> > > user@tomee:~/haproxy-1.6.5$ gmake TARGET=solaris
>> > > gcc -Iinclude -Iebtree -Wall -O2 -g -fno-strict-aliasing -
>> > > Wdeclaration-after-statement -fomit-frame-pointer -
>> > DFD_SETSIZE=65536 -
>> > > D_REENTRANT -DTPROXY -DCONFIG_HAP_CRYPT -DNEED_CRYPT_H -
>> > > DUSE_GETADDRINFO -DENABLE_POLL -
>> > DCONFIG_HAPROXY_VERSION=\"1.6.5\" -
>> > > DCONFIG_HAPROXY_DATE=\"2016/05/10\" \
>> > > -DBUILD_TARGET='"solaris"' \
>> > > -DBUILD_ARCH='""' \
>> > > -DBUILD_CPU='"generic"' \
>> > > -DBUILD_CC='"gcc"' \
>> > > -DBUILD_CFLAGS='"-O2 -g -fno-strict-aliasing -
>> > > Wdeclaration-after-statement -fomit-frame-pointer -
>> > DFD_SETSIZE=65536 -
>> > > D_REENTRANT"' \
>> > > -DBUILD_OPTIONS='""' \
>> > > -c -o src/haproxy.o src/haproxy.c
>> > > In file included from include/common/ticks.h:56:0,
>> > > from include/proto/proxy.h:26,
>> > > from include/common/cfgparse.h:30,
>> > > from src/haproxy.c:63:
>> > > include/common/standard.h:1013:34: error: static declaration of
>> > > 'htonll' follows non-static declaration
>> > > static inline unsigned long long htonll(unsigned long long a)
>> > > ^
>> > > In file included from /usr/include/netinet/in.h:73:0,
>> > > from /usr/include/sys/socket.h:29,
>> > > from src/haproxy.c:36:
>> > > /usr/include/sys/byteorder.h:75:17: note: previous declaration
>> > of
>> > > 'htonll' was here
>> > > extern uint64_t htonll(uint64_t);
>> > > ^
>> > > In file included from include/common/ticks.h:56:0,
>> > > from include/proto/proxy.h:26,
>> > > from include/common/cfgparse.h:30,
>> > > from src/haproxy.c:63:
>> > > include/common/standard.h:1028:34: error: static declaration of
>> > > 'ntohll' follows non-static declaration
>> > > static inline unsigned long long ntohll(unsigned long long a)
>> > > ^
>> > > In file included from /usr/include/netinet/in.h:73:0,
>> > > from /usr/include/sys/socket.h:29,
>> > > from src/haproxy.c:36:
>> > > /usr/include/sys/byteorder.h:76:17: note: previous declaration
>> > of
>> > > 'ntohll' was here
>> > > extern uint64_t ntohll(uint64_t);
>> > > ^
>> > > gmake: *** [src/haproxy.o] Error 1
>> > > user@tomee:~/haproxy-1.6.5$
>> > >
>> > > cheers,
>>
>>
>
>
> --
>
> *Jonathan S. Fisher*
> Senior Software Engineer
> https://twitter.com/exabrial
> http://www.tomitribe.com
> https://www.tomitribe.io
>



-- 

*Jonathan S. Fisher*
Senior 

Re: [PATCH] BUG/MAJOR: fix listening IP address storage for frontends

[Cyril Bonté]

Hi,

Le 18/05/2016 22:36, Arthur Țițeică a écrit :

Hi all,

În ziua de miercuri, 18 mai 2016, la 20:51:13 EEST, Willy Tarreau a scris:

Thanks Vincent!

It looks pretty good and very clean in the end.
Arthur, as soon as you confirm it works for you I'll merge it. I'm keeping
it untouched below in case you missed it.


Something seems a bit off now.

This is what happens when I manually start haproxy with the patch applied

# /usr/bin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:25] : unknown
keyword 'tune.ssl.default-dh-param' in 'global' section
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:26] : 'ssl-
default-bind-ciphers' is not implemented.
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:27] : unknown
keyword 'ssl-default-bind-options' in 'global' section
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:28] : 'ssl-
default-server-ciphers' is not implemented.
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:29] : unknown
keyword 'ssl-default-server-options' in 'global' section
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:31] : 'crt-
base' is not implemented.
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:78] : 'bind
163.172.36.33:443' unknown keyword 'ssl'. Registered keywords :
 [ ALL] accept-proxy
 [ ALL] backlog 
 [ ALL] id 
 [ ALL] maxconn 
 [ ALL] name 
 [ ALL] nice 
 [ ALL] process 
 [UNIX] gid 
 [UNIX] group 
 [UNIX] mode 
 [UNIX] uid 
 [UNIX] user 
 [STAT] level 
 [ TCP] defer-accept
 [ TCP] interface 
 [ TCP] mss 
 [ TCP] tcp-ut 
 [ TCP] tfo
 [ TCP] transparent
 [ TCP] v4v6
 [ TCP] v6only
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:79] : 'bind
2001:bc8:2377:200::1:443' unknown keyword 'ssl'.
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:80] : 'bind
163.172.36.33:59091' unknown keyword 'ssl'.
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:81] : 'bind
2001:bc8:2377:200::1:59091' unknown keyword 'ssl'.
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:82] : 'bind
163.172.36.33:59092' unknown keyword 'ssl'.
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:83] : 'bind
2001:bc8:2377:200::1:59092' unknown keyword 'ssl'.
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:84] : 'bind
163.172.36.33:8099' unknown keyword 'ssl'.
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:85] : 'bind
2001:bc8:2377:200::1:8099' unknown keyword 'ssl'.
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:142] : error
detected while parsing an 'http-request auth' condition : unknown fetch method
'ssl_fc' in ACL expression 'ssl_fc'.
[ALERT] 138/232913 (14342) : Error(s) found in configuration file : /etc/
haproxy/haproxy.cfg

Needless to say that this is a valid config that works with 1.6.4.


It looks like you didn't recompile with USE_OPENSSL=1
haproxy -vv should give some hints.

--
Cyril Bonté

Re: [PATCH] BUG/MAJOR: fix listening IP address storage for frontends

[Arthur Țițeică]

Hi all,

În ziua de miercuri, 18 mai 2016, la 20:51:13 EEST, Willy Tarreau a scris:
> Thanks Vincent!
> 
> It looks pretty good and very clean in the end.
> Arthur, as soon as you confirm it works for you I'll merge it. I'm keeping
> it untouched below in case you missed it.

Something seems a bit off now.

This is what happens when I manually start haproxy with the patch applied

# /usr/bin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:25] : unknown 
keyword 'tune.ssl.default-dh-param' in 'global' section
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:26] : 'ssl-
default-bind-ciphers' is not implemented.
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:27] : unknown 
keyword 'ssl-default-bind-options' in 'global' section
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:28] : 'ssl-
default-server-ciphers' is not implemented.
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:29] : unknown 
keyword 'ssl-default-server-options' in 'global' section
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:31] : 'crt-
base' is not implemented.
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:78] : 'bind 
163.172.36.33:443' unknown keyword 'ssl'. Registered keywords :
[ ALL] accept-proxy
[ ALL] backlog 
[ ALL] id 
[ ALL] maxconn 
[ ALL] name 
[ ALL] nice 
[ ALL] process 
[UNIX] gid 
[UNIX] group 
[UNIX] mode 
[UNIX] uid 
[UNIX] user 
[STAT] level 
[ TCP] defer-accept
[ TCP] interface 
[ TCP] mss 
[ TCP] tcp-ut 
[ TCP] tfo
[ TCP] transparent
[ TCP] v4v6
[ TCP] v6only
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:79] : 'bind 
2001:bc8:2377:200::1:443' unknown keyword 'ssl'.
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:80] : 'bind 
163.172.36.33:59091' unknown keyword 'ssl'.
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:81] : 'bind 
2001:bc8:2377:200::1:59091' unknown keyword 'ssl'.
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:82] : 'bind 
163.172.36.33:59092' unknown keyword 'ssl'.
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:83] : 'bind 
2001:bc8:2377:200::1:59092' unknown keyword 'ssl'.
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:84] : 'bind 
163.172.36.33:8099' unknown keyword 'ssl'.
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:85] : 'bind 
2001:bc8:2377:200::1:8099' unknown keyword 'ssl'.
[ALERT] 138/232913 (14342) : parsing [/etc/haproxy/haproxy.cfg:142] : error 
detected while parsing an 'http-request auth' condition : unknown fetch method 
'ssl_fc' in ACL expression 'ssl_fc'.
[ALERT] 138/232913 (14342) : Error(s) found in configuration file : /etc/
haproxy/haproxy.cfg

Needless to say that this is a valid config that works with 1.6.4.

Thanks

Re: [PATCH] BUG/MAJOR: fix listening IP address storage for frontends

[Willy Tarreau]

Thanks Vincent!

It looks pretty good and very clean in the end.
Arthur, as soon as you confirm it works for you I'll merge it. I'm keeping
it untouched below in case you missed it.

Thanks,
Willy

On Wed, May 18, 2016 at 04:17:44PM +0200, Vincent Bernat wrote:
> From: Vincent Bernat 
> 
> When compiled with GCC 6, the IP address specified for a frontend was
> ignored and HAProxy was listening on all addresses instead. This is
> caused by an incomplete copy of a "struct sockaddr_storage".
> 
> With the GNU Libc, "struct sockaddr_storage" is defined as this:
> 
> struct sockaddr_storage
>   {
> sa_family_t ss_family;
> unsigned long int __ss_align;
> char __ss_padding[(128 - (2 * sizeof (unsigned long int)))];
>   };
> 
> Doing an aggregate copy (ss1 = ss2) is different than using memcpy():
> only members of the aggregate have to be copied. Notably, padding can be
> or not be copied. In GCC 6, some optimizations use this fact and if a
> "struct sockaddr_storage" contains a "struct sockaddr_in", the port and
> the address are part of the padding (between sa_family and __ss_align)
> and can be not copied over.
> 
> Therefore, we replace any aggregate copy by a memcpy(). There is another
> place using the same pattern. We also fix a function receiving a "struct
> sockaddr_storage" by copy instead of by reference. Since it only needs a
> read-only copy, the function is converted to request a reference.
> ---
>  include/proto/proto_http.h |  2 +-
>  src/cfgparse.c |  2 +-
>  src/connection.c   |  2 +-
>  src/hlua.c |  2 +-
>  src/proto_http.c   | 12 ++--
>  src/proto_tcp.c|  2 +-
>  6 files changed, 11 insertions(+), 11 deletions(-)
> 
> diff --git a/include/proto/proto_http.h b/include/proto/proto_http.h
> index 4d8f5365b625..0aa6643b98da 100644
> --- a/include/proto/proto_http.h
> +++ b/include/proto/proto_http.h
> @@ -110,7 +110,7 @@ void http_set_status(unsigned int status, struct stream 
> *s);
>  int http_transform_header_str(struct stream* s, struct http_msg *msg, const 
> char* name,
>unsigned int name_len, const char *str, struct 
> my_regex *re,
>int action);
> -void inet_set_tos(int fd, struct sockaddr_storage from, int tos);
> +void inet_set_tos(int fd, const struct sockaddr_storage *from, int tos);
>  void http_perform_server_redirect(struct stream *s, struct stream_interface 
> *si);
>  void http_return_srv_error(struct stream *s, struct stream_interface *si);
>  void http_capture_bad_message(struct error_snapshot *es, struct stream *s,
> diff --git a/src/cfgparse.c b/src/cfgparse.c
> index 3fee54e0db1d..48e584cf73e7 100644
> --- a/src/cfgparse.c
> +++ b/src/cfgparse.c
> @@ -287,7 +287,7 @@ int str2listener(char *str, struct proxy *curproxy, 
> struct bind_conf *bind_conf,
>   }
>  
>   /* OK the address looks correct */
> - ss = *ss2;
> + memcpy(, ss2, sizeof(ss));
>  
>   for (; port <= end; port++) {
>   l = calloc(1, sizeof(*l));
> diff --git a/src/connection.c b/src/connection.c
> index 330f3efbc995..5515188c6b10 100644
> --- a/src/connection.c
> +++ b/src/connection.c
> @@ -744,7 +744,7 @@ int make_proxy_line_v2(char *buf, int buf_len, struct 
> server *srv, struct connec
>   const char pp2_signature[] = PP2_SIGNATURE;
>   int ret = 0;
>   struct proxy_hdr_v2 *hdr = (struct proxy_hdr_v2 *)buf;
> - struct sockaddr_storage null_addr = {0};
> + struct sockaddr_storage null_addr = { .ss_family = 0 };
>   struct sockaddr_storage *src = _addr;
>   struct sockaddr_storage *dst = _addr;
>  
> diff --git a/src/hlua.c b/src/hlua.c
> index f6eb8aa80ee0..94f97429c895 100644
> --- a/src/hlua.c
> +++ b/src/hlua.c
> @@ -4781,7 +4781,7 @@ __LJMP static int hlua_txn_set_tos(lua_State *L)
>   tos = MAY_LJMP(luaL_checkinteger(L, 2));
>  
>   if ((cli_conn = objt_conn(htxn->s->sess->origin)) && 
> conn_ctrl_ready(cli_conn))
> - inet_set_tos(cli_conn->t.sock.fd, cli_conn->addr.from, tos);
> + inet_set_tos(cli_conn->t.sock.fd, _conn->addr.from, tos);
>  
>   return 0;
>  }
> diff --git a/src/proto_http.c b/src/proto_http.c
> index 21ad131c9f43..416504247a8d 100644
> --- a/src/proto_http.c
> +++ b/src/proto_http.c
> @@ -3189,15 +3189,15 @@ int http_handle_stats(struct stream *s, struct 
> channel *req)
>  /* Sets the TOS header in IPv4 and the traffic class header in IPv6 packets
>   * (as per RFC3260 #4 and BCP37 #4.2 and #5.2).
>   */
> -void inet_set_tos(int fd, struct sockaddr_storage from, int tos)
> +void inet_set_tos(int fd, const struct sockaddr_storage *from, int tos)
>  {
>  #ifdef IP_TOS
> - if (from.ss_family == AF_INET)
> + if (from->ss_family == AF_INET)
>   setsockopt(fd, IPPROTO_IP, IP_TOS, , sizeof(tos));
>  #endif
>  #ifdef IPV6_TCLASS
> - if 

Re: Compilation problem: haproxy 1.6.5 (latest) on Solaris 11

[Jonathan Fisher]

Nice here's the complication output:


http://pastebin.com/iS2JKXED

Now I just have to figure out how to add openssl, zlib, and libpcre which
don't seem to be available on Oracle Solaris.

On Wed, May 18, 2016 at 1:40 PM, Vincent Bernat  wrote:

> You can try this patch to check if it works.
>
>
>
> --
> Let the data structure the program.
> - The Elements of Programming Style (Kernighan & Plauger)
>
>  ――― Original Message ―――
>  From: Jonathan Fisher 
>  Sent: 18 mai 2016 12:56 -0400
>  Subject: Re: Compilation problem: haproxy 1.6.5 (latest) on Solaris 11
>  To: Vincent Bernat
>  Cc: haproxy@formilux.org
>
> > Oh man, I wish I was smart enough to understand what's happening
> > there... what do I need to replace? (And for my own learning/benefit,
> > can you explain what's happening?)
> >
> > Thank you!!!
> >
> > On Wed, May 18, 2016 at 12:07 PM, Vincent Bernat 
> > wrote:
> >
> > Hey!
> >
> > Since there is some discrepancy in the definition of ntohll among
> > platforms, I define this macro to shadow any existing definition:
> >
> > #ifndef ntohll
> > # define ntohll(x) \
> > (((u_int64_t)(ntohl((int)(((x) << 32) >> 32))) << 32) | \
> > (unsigned int)ntohl(((int)((x) >> 32
> > #endif
> >
> > You can try yourself. Or if you don't know what to replace, I can
> > provide a patch.
> > --
> > Don't compare floating point numbers just for equality.
> > - The Elements of Programming Style (Kernighan & Plauger)
> >
> > ――― Original Message ―――
> > From: Jonathan Fisher 
> > Sent: 18 mai 2016 10:22 -0400
> > Subject: Compilation problem: haproxy 1.6.5 (latest) on Solaris 11
> > To: haproxy@formilux.org
> >
> >
> >
> > > We can get 1.5 to compile quite nicely on Solaris 11 :) 1.6 has
> > a few
> > > nice features though we'd like to have, if it all possible. I
> > have a
> > > VMDK created if the developers want to try it themselves in a
> > virtual
> > > machine.
> > >
> > > Here is the output of the compiler, which is GCC-48:
> > >
> > > user@tomee:~/haproxy-1.6.5$ gmake TARGET=solaris
> > > gcc -Iinclude -Iebtree -Wall -O2 -g -fno-strict-aliasing -
> > > Wdeclaration-after-statement -fomit-frame-pointer -
> > DFD_SETSIZE=65536 -
> > > D_REENTRANT -DTPROXY -DCONFIG_HAP_CRYPT -DNEED_CRYPT_H -
> > > DUSE_GETADDRINFO -DENABLE_POLL -
> > DCONFIG_HAPROXY_VERSION=\"1.6.5\" -
> > > DCONFIG_HAPROXY_DATE=\"2016/05/10\" \
> > > -DBUILD_TARGET='"solaris"' \
> > > -DBUILD_ARCH='""' \
> > > -DBUILD_CPU='"generic"' \
> > > -DBUILD_CC='"gcc"' \
> > > -DBUILD_CFLAGS='"-O2 -g -fno-strict-aliasing -
> > > Wdeclaration-after-statement -fomit-frame-pointer -
> > DFD_SETSIZE=65536 -
> > > D_REENTRANT"' \
> > > -DBUILD_OPTIONS='""' \
> > > -c -o src/haproxy.o src/haproxy.c
> > > In file included from include/common/ticks.h:56:0,
> > > from include/proto/proxy.h:26,
> > > from include/common/cfgparse.h:30,
> > > from src/haproxy.c:63:
> > > include/common/standard.h:1013:34: error: static declaration of
> > > 'htonll' follows non-static declaration
> > > static inline unsigned long long htonll(unsigned long long a)
> > > ^
> > > In file included from /usr/include/netinet/in.h:73:0,
> > > from /usr/include/sys/socket.h:29,
> > > from src/haproxy.c:36:
> > > /usr/include/sys/byteorder.h:75:17: note: previous declaration
> > of
> > > 'htonll' was here
> > > extern uint64_t htonll(uint64_t);
> > > ^
> > > In file included from include/common/ticks.h:56:0,
> > > from include/proto/proxy.h:26,
> > > from include/common/cfgparse.h:30,
> > > from src/haproxy.c:63:
> > > include/common/standard.h:1028:34: error: static declaration of
> > > 'ntohll' follows non-static declaration
> > > static inline unsigned long long ntohll(unsigned long long a)
> > > ^
> > > In file included from /usr/include/netinet/in.h:73:0,
> > > from /usr/include/sys/socket.h:29,
> > > from src/haproxy.c:36:
> > > /usr/include/sys/byteorder.h:76:17: note: previous declaration
> > of
> > > 'ntohll' was here
> > > extern uint64_t ntohll(uint64_t);
> > > ^
> > > gmake: *** [src/haproxy.o] Error 1
> > > user@tomee:~/haproxy-1.6.5$
> > >
> > > cheers,
>
>


-- 

*Jonathan S. Fisher*
Senior Software Engineer
https://twitter.com/exabrial
http://www.tomitribe.com
https://www.tomitribe.io

Re: Compilation problem: haproxy 1.6.5 (latest) on Solaris 11

[Vincent Bernat]

You can try this patch to check if it works.

>From 74b502bcbef74927b2e006ac399a4d3b3de1d331 Mon Sep 17 00:00:00 2001
From: Vincent Bernat 
Date: Wed, 18 May 2016 19:38:36 +0200
Subject: [PATCH] MINOR: use a macro for htonll/ntohll

Some OS have a definition for htonll/ntohll using a regular function and
the signature is not compatible with the provided function. Instead,
provide a macro that would shadow such a definition instead of
conflicting with it.
---
 include/common/standard.h | 8 +++-
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/include/common/standard.h b/include/common/standard.h
index f123f1a41dfb..2cd7fc48937c 100644
--- a/include/common/standard.h
+++ b/include/common/standard.h
@@ -1049,7 +1049,7 @@ static inline unsigned char utf8_return_length(unsigned char code)
  * a few bswap and 2 shifts are left, which is the minimum acceptable.
  */
 #ifndef htonll
-static inline unsigned long long htonll(unsigned long long a)
+static inline unsigned long long _htonll(unsigned long long a)
 {
 	union {
 		struct {
@@ -1060,14 +1060,12 @@ static inline unsigned long long htonll(unsigned long long a)
 	} w = { .by64 = a };
 	return ((unsigned long long)htonl(w.by32.w1) << 32) | htonl(w.by32.w2);
 }
+#define htonll(a) _htonll(a)
 #endif
 
 /* Turns 64-bit value  from network byte order to host byte order. */
 #ifndef ntohll
-static inline unsigned long long ntohll(unsigned long long a)
-{
-	return htonll(a);
-}
+#define ntohll(a) htonll(a)
 #endif
 
 /* returns a 64-bit a timestamp with the finest resolution available. The
-- 
2.8.1


-- 
Let the data structure the program.
- The Elements of Programming Style (Kernighan & Plauger)

 ――― Original Message ―――
 From: Jonathan Fisher 
 Sent: 18 mai 2016 12:56 -0400
 Subject: Re: Compilation problem: haproxy 1.6.5 (latest) on Solaris 11
 To: Vincent Bernat
 Cc: haproxy@formilux.org

> Oh man, I wish I was smart enough to understand what's happening
> there... what do I need to replace? (And for my own learning/benefit,
> can you explain what's happening?)
>
> Thank you!!!
>
> On Wed, May 18, 2016 at 12:07 PM, Vincent Bernat 
> wrote:
>
> Hey!
> 
> Since there is some discrepancy in the definition of ntohll among
> platforms, I define this macro to shadow any existing definition:
> 
> #ifndef ntohll
> # define ntohll(x) \
> (((u_int64_t)(ntohl((int)(((x) << 32) >> 32))) << 32) | \
> (unsigned int)ntohl(((int)((x) >> 32
> #endif
> 
> You can try yourself. Or if you don't know what to replace, I can
> provide a patch.
> --
> Don't compare floating point numbers just for equality.
> - The Elements of Programming Style (Kernighan & Plauger)
> 
> ――― Original Message ―――
> From: Jonathan Fisher 
> Sent: 18 mai 2016 10:22 -0400
> Subject: Compilation problem: haproxy 1.6.5 (latest) on Solaris 11
> To: haproxy@formilux.org
> 
> 
> 
> > We can get 1.5 to compile quite nicely on Solaris 11 :) 1.6 has
> a few
> > nice features though we'd like to have, if it all possible. I
> have a
> > VMDK created if the developers want to try it themselves in a
> virtual
> > machine.
> >
> > Here is the output of the compiler, which is GCC-48:
> >
> > user@tomee:~/haproxy-1.6.5$ gmake TARGET=solaris
> > gcc -Iinclude -Iebtree -Wall -O2 -g -fno-strict-aliasing -
> > Wdeclaration-after-statement -fomit-frame-pointer -
> DFD_SETSIZE=65536 -
> > D_REENTRANT -DTPROXY -DCONFIG_HAP_CRYPT -DNEED_CRYPT_H -
> > DUSE_GETADDRINFO -DENABLE_POLL -
> DCONFIG_HAPROXY_VERSION=\"1.6.5\" -
> > DCONFIG_HAPROXY_DATE=\"2016/05/10\" \
> > -DBUILD_TARGET='"solaris"' \
> > -DBUILD_ARCH='""' \
> > -DBUILD_CPU='"generic"' \
> > -DBUILD_CC='"gcc"' \
> > -DBUILD_CFLAGS='"-O2 -g -fno-strict-aliasing -
> > Wdeclaration-after-statement -fomit-frame-pointer -
> DFD_SETSIZE=65536 -
> > D_REENTRANT"' \
> > -DBUILD_OPTIONS='""' \
> > -c -o src/haproxy.o src/haproxy.c
> > In file included from include/common/ticks.h:56:0,
> > from include/proto/proxy.h:26,
> > from include/common/cfgparse.h:30,
> > from src/haproxy.c:63:
> > include/common/standard.h:1013:34: error: static declaration of
> > 'htonll' follows non-static declaration
> > static inline unsigned long long htonll(unsigned long long a)
> > ^
> > In file included from /usr/include/netinet/in.h:73:0,
> > from /usr/include/sys/socket.h:29,
> > from src/haproxy.c:36:
> > /usr/include/sys/byteorder.h:75:17: note: previous declaration
> of
> > 'htonll' was here
> > extern uint64_t htonll(uint64_t);
> > ^
> > In file included from include/common/ticks.h:56:0,
> > from include/proto/proxy.h:26,
> > from include/common/cfgparse.h:30,
> > from 

Re: Compilation problem: haproxy 1.6.5 (latest) on Solaris 11

[Jonathan Fisher]

Oh man, I wish I was smart enough to understand what's happening there...
what do I need to replace? (And for my own learning/benefit, can you
explain what's happening?)

Thank you!!!

On Wed, May 18, 2016 at 12:07 PM, Vincent Bernat  wrote:

> Hey!
>
> Since there is some discrepancy in the definition of ntohll among
> platforms, I define this macro to shadow any existing definition:
>
> #ifndef ntohll
> # define ntohll(x)  \
> (((u_int64_t)(ntohl((int)(((x) << 32) >> 32))) << 32) | \
> (unsigned int)ntohl(((int)((x) >> 32
> #endif
>
> You can try yourself. Or if you don't know what to replace, I can
> provide a patch.
> --
> Don't compare floating point numbers just for equality.
> - The Elements of Programming Style (Kernighan & Plauger)
>
>  ――― Original Message ―――
>  From: Jonathan Fisher 
>  Sent: 18 mai 2016 10:22 -0400
>  Subject: Compilation problem: haproxy 1.6.5 (latest) on Solaris 11
>  To: haproxy@formilux.org
>
> > We can get 1.5 to compile quite nicely on Solaris 11 :) 1.6 has a few
> > nice features though we'd like to have, if it all possible. I have a
> > VMDK created if the developers want to try it themselves in a virtual
> > machine.
> >
> > Here is the output of the compiler, which is GCC-48:
> >
> > user@tomee:~/haproxy-1.6.5$ gmake TARGET=solaris
> > gcc -Iinclude -Iebtree -Wall -O2 -g -fno-strict-aliasing -
> > Wdeclaration-after-statement -fomit-frame-pointer -DFD_SETSIZE=65536 -
> > D_REENTRANT -DTPROXY -DCONFIG_HAP_CRYPT -DNEED_CRYPT_H -
> > DUSE_GETADDRINFO -DENABLE_POLL -DCONFIG_HAPROXY_VERSION=\"1.6.5\" -
> > DCONFIG_HAPROXY_DATE=\"2016/05/10\" \
> > -DBUILD_TARGET='"solaris"' \
> > -DBUILD_ARCH='""' \
> > -DBUILD_CPU='"generic"' \
> > -DBUILD_CC='"gcc"' \
> > -DBUILD_CFLAGS='"-O2 -g -fno-strict-aliasing -
> > Wdeclaration-after-statement -fomit-frame-pointer -DFD_SETSIZE=65536 -
> > D_REENTRANT"' \
> > -DBUILD_OPTIONS='""' \
> > -c -o src/haproxy.o src/haproxy.c
> > In file included from include/common/ticks.h:56:0,
> > from include/proto/proxy.h:26,
> > from include/common/cfgparse.h:30,
> > from src/haproxy.c:63:
> > include/common/standard.h:1013:34: error: static declaration of
> > 'htonll' follows non-static declaration
> > static inline unsigned long long htonll(unsigned long long a)
> > ^
> > In file included from /usr/include/netinet/in.h:73:0,
> > from /usr/include/sys/socket.h:29,
> > from src/haproxy.c:36:
> > /usr/include/sys/byteorder.h:75:17: note: previous declaration of
> > 'htonll' was here
> > extern uint64_t htonll(uint64_t);
> > ^
> > In file included from include/common/ticks.h:56:0,
> > from include/proto/proxy.h:26,
> > from include/common/cfgparse.h:30,
> > from src/haproxy.c:63:
> > include/common/standard.h:1028:34: error: static declaration of
> > 'ntohll' follows non-static declaration
> > static inline unsigned long long ntohll(unsigned long long a)
> > ^
> > In file included from /usr/include/netinet/in.h:73:0,
> > from /usr/include/sys/socket.h:29,
> > from src/haproxy.c:36:
> > /usr/include/sys/byteorder.h:76:17: note: previous declaration of
> > 'ntohll' was here
> > extern uint64_t ntohll(uint64_t);
> > ^
> > gmake: *** [src/haproxy.o] Error 1
> > user@tomee:~/haproxy-1.6.5$
> >
> > cheers,
>



-- 

*Jonathan S. Fisher*
Senior Software Engineer
https://twitter.com/exabrial
http://www.tomitribe.com
https://www.tomitribe.io

Re: Compilation problem: haproxy 1.6.5 (latest) on Solaris 11

[Vincent Bernat]

Hey!

Since there is some discrepancy in the definition of ntohll among
platforms, I define this macro to shadow any existing definition:

#ifndef ntohll
# define ntohll(x)  \
(((u_int64_t)(ntohl((int)(((x) << 32) >> 32))) << 32) | \
(unsigned int)ntohl(((int)((x) >> 32
#endif

You can try yourself. Or if you don't know what to replace, I can
provide a patch.
-- 
Don't compare floating point numbers just for equality.
- The Elements of Programming Style (Kernighan & Plauger)

 ――― Original Message ―――
 From: Jonathan Fisher 
 Sent: 18 mai 2016 10:22 -0400
 Subject: Compilation problem: haproxy 1.6.5 (latest) on Solaris 11
 To: haproxy@formilux.org

> We can get 1.5 to compile quite nicely on Solaris 11 :) 1.6 has a few
> nice features though we'd like to have, if it all possible. I have a
> VMDK created if the developers want to try it themselves in a virtual
> machine.
>
> Here is the output of the compiler, which is GCC-48:
>
> user@tomee:~/haproxy-1.6.5$ gmake TARGET=solaris
> gcc -Iinclude -Iebtree -Wall -O2 -g -fno-strict-aliasing -
> Wdeclaration-after-statement -fomit-frame-pointer -DFD_SETSIZE=65536 -
> D_REENTRANT -DTPROXY -DCONFIG_HAP_CRYPT -DNEED_CRYPT_H -
> DUSE_GETADDRINFO -DENABLE_POLL -DCONFIG_HAPROXY_VERSION=\"1.6.5\" -
> DCONFIG_HAPROXY_DATE=\"2016/05/10\" \
> -DBUILD_TARGET='"solaris"' \
> -DBUILD_ARCH='""' \
> -DBUILD_CPU='"generic"' \
> -DBUILD_CC='"gcc"' \
> -DBUILD_CFLAGS='"-O2 -g -fno-strict-aliasing -
> Wdeclaration-after-statement -fomit-frame-pointer -DFD_SETSIZE=65536 -
> D_REENTRANT"' \
> -DBUILD_OPTIONS='""' \
> -c -o src/haproxy.o src/haproxy.c
> In file included from include/common/ticks.h:56:0,
> from include/proto/proxy.h:26,
> from include/common/cfgparse.h:30,
> from src/haproxy.c:63:
> include/common/standard.h:1013:34: error: static declaration of
> 'htonll' follows non-static declaration
> static inline unsigned long long htonll(unsigned long long a)
> ^
> In file included from /usr/include/netinet/in.h:73:0,
> from /usr/include/sys/socket.h:29,
> from src/haproxy.c:36:
> /usr/include/sys/byteorder.h:75:17: note: previous declaration of
> 'htonll' was here
> extern uint64_t htonll(uint64_t);
> ^
> In file included from include/common/ticks.h:56:0,
> from include/proto/proxy.h:26,
> from include/common/cfgparse.h:30,
> from src/haproxy.c:63:
> include/common/standard.h:1028:34: error: static declaration of
> 'ntohll' follows non-static declaration
> static inline unsigned long long ntohll(unsigned long long a)
> ^
> In file included from /usr/include/netinet/in.h:73:0,
> from /usr/include/sys/socket.h:29,
> from src/haproxy.c:36:
> /usr/include/sys/byteorder.h:76:17: note: previous declaration of
> 'ntohll' was here
> extern uint64_t ntohll(uint64_t);
> ^
> gmake: *** [src/haproxy.o] Error 1
> user@tomee:~/haproxy-1.6.5$ 
>
> cheers,

Compilation problem: haproxy 1.6.5 (latest) on Solaris 11

[Jonathan Fisher]

We can get 1.5 to compile quite nicely on Solaris 11 :) 1.6 has a few nice
features though we'd like to have, if it all possible. I have a VMDK
created if the developers want to try it themselves in a virtual machine.

Here is the output of the compiler, which is GCC-48:

user@tomee:~/haproxy-1.6.5$ gmake TARGET=solaris
gcc -Iinclude -Iebtree -Wall  -O2 -g -fno-strict-aliasing
-Wdeclaration-after-statement -fomit-frame-pointer -DFD_SETSIZE=65536
-D_REENTRANT  -DTPROXY -DCONFIG_HAP_CRYPT -DNEED_CRYPT_H
-DUSE_GETADDRINFO -DENABLE_POLL  -DCONFIG_HAPROXY_VERSION=\"1.6.5\"
-DCONFIG_HAPROXY_DATE=\"2016/05/10\" \
  -DBUILD_TARGET='"solaris"' \
  -DBUILD_ARCH='""' \
  -DBUILD_CPU='"generic"' \
  -DBUILD_CC='"gcc"' \
  -DBUILD_CFLAGS='"-O2 -g -fno-strict-aliasing
-Wdeclaration-after-statement -fomit-frame-pointer -DFD_SETSIZE=65536
-D_REENTRANT"' \
  -DBUILD_OPTIONS='""' \
   -c -o src/haproxy.o src/haproxy.c
In file included from include/common/ticks.h:56:0,
 from include/proto/proxy.h:26,
 from include/common/cfgparse.h:30,
 from src/haproxy.c:63:
include/common/standard.h:1013:34: error: static declaration of 'htonll'
follows non-static declaration
 static inline unsigned long long htonll(unsigned long long a)
  ^
In file included from /usr/include/netinet/in.h:73:0,
 from /usr/include/sys/socket.h:29,
 from src/haproxy.c:36:
/usr/include/sys/byteorder.h:75:17: note: previous declaration of 'htonll'
was here
 extern uint64_t htonll(uint64_t);
 ^
In file included from include/common/ticks.h:56:0,
 from include/proto/proxy.h:26,
 from include/common/cfgparse.h:30,
 from src/haproxy.c:63:
include/common/standard.h:1028:34: error: static declaration of 'ntohll'
follows non-static declaration
 static inline unsigned long long ntohll(unsigned long long a)
  ^
In file included from /usr/include/netinet/in.h:73:0,
 from /usr/include/sys/socket.h:29,
 from src/haproxy.c:36:
/usr/include/sys/byteorder.h:76:17: note: previous declaration of 'ntohll'
was here
 extern uint64_t ntohll(uint64_t);
 ^
gmake: *** [src/haproxy.o] Error 1
user@tomee:~/haproxy-1.6.5$


cheers,

-- 

*Jonathan S. Fisher*
Senior Software Engineer
https://twitter.com/exabrial
http://www.tomitribe.com
https://www.tomitribe.io

[PATCH] BUG/MAJOR: fix listening IP address storage for frontends

[Vincent Bernat]

From: Vincent Bernat 

When compiled with GCC 6, the IP address specified for a frontend was
ignored and HAProxy was listening on all addresses instead. This is
caused by an incomplete copy of a "struct sockaddr_storage".

With the GNU Libc, "struct sockaddr_storage" is defined as this:

struct sockaddr_storage
  {
sa_family_t ss_family;
unsigned long int __ss_align;
char __ss_padding[(128 - (2 * sizeof (unsigned long int)))];
  };

Doing an aggregate copy (ss1 = ss2) is different than using memcpy():
only members of the aggregate have to be copied. Notably, padding can be
or not be copied. In GCC 6, some optimizations use this fact and if a
"struct sockaddr_storage" contains a "struct sockaddr_in", the port and
the address are part of the padding (between sa_family and __ss_align)
and can be not copied over.

Therefore, we replace any aggregate copy by a memcpy(). There is another
place using the same pattern. We also fix a function receiving a "struct
sockaddr_storage" by copy instead of by reference. Since it only needs a
read-only copy, the function is converted to request a reference.
---
 include/proto/proto_http.h |  2 +-
 src/cfgparse.c |  2 +-
 src/connection.c   |  2 +-
 src/hlua.c |  2 +-
 src/proto_http.c   | 12 ++--
 src/proto_tcp.c|  2 +-
 6 files changed, 11 insertions(+), 11 deletions(-)

diff --git a/include/proto/proto_http.h b/include/proto/proto_http.h
index 4d8f5365b625..0aa6643b98da 100644
--- a/include/proto/proto_http.h
+++ b/include/proto/proto_http.h
@@ -110,7 +110,7 @@ void http_set_status(unsigned int status, struct stream *s);
 int http_transform_header_str(struct stream* s, struct http_msg *msg, const 
char* name,
   unsigned int name_len, const char *str, struct 
my_regex *re,
   int action);
-void inet_set_tos(int fd, struct sockaddr_storage from, int tos);
+void inet_set_tos(int fd, const struct sockaddr_storage *from, int tos);
 void http_perform_server_redirect(struct stream *s, struct stream_interface 
*si);
 void http_return_srv_error(struct stream *s, struct stream_interface *si);
 void http_capture_bad_message(struct error_snapshot *es, struct stream *s,
diff --git a/src/cfgparse.c b/src/cfgparse.c
index 3fee54e0db1d..48e584cf73e7 100644
--- a/src/cfgparse.c
+++ b/src/cfgparse.c
@@ -287,7 +287,7 @@ int str2listener(char *str, struct proxy *curproxy, struct 
bind_conf *bind_conf,
}
 
/* OK the address looks correct */
-   ss = *ss2;
+   memcpy(, ss2, sizeof(ss));
 
for (; port <= end; port++) {
l = calloc(1, sizeof(*l));
diff --git a/src/connection.c b/src/connection.c
index 330f3efbc995..5515188c6b10 100644
--- a/src/connection.c
+++ b/src/connection.c
@@ -744,7 +744,7 @@ int make_proxy_line_v2(char *buf, int buf_len, struct 
server *srv, struct connec
const char pp2_signature[] = PP2_SIGNATURE;
int ret = 0;
struct proxy_hdr_v2 *hdr = (struct proxy_hdr_v2 *)buf;
-   struct sockaddr_storage null_addr = {0};
+   struct sockaddr_storage null_addr = { .ss_family = 0 };
struct sockaddr_storage *src = _addr;
struct sockaddr_storage *dst = _addr;
 
diff --git a/src/hlua.c b/src/hlua.c
index f6eb8aa80ee0..94f97429c895 100644
--- a/src/hlua.c
+++ b/src/hlua.c
@@ -4781,7 +4781,7 @@ __LJMP static int hlua_txn_set_tos(lua_State *L)
tos = MAY_LJMP(luaL_checkinteger(L, 2));
 
if ((cli_conn = objt_conn(htxn->s->sess->origin)) && 
conn_ctrl_ready(cli_conn))
-   inet_set_tos(cli_conn->t.sock.fd, cli_conn->addr.from, tos);
+   inet_set_tos(cli_conn->t.sock.fd, _conn->addr.from, tos);
 
return 0;
 }
diff --git a/src/proto_http.c b/src/proto_http.c
index 21ad131c9f43..416504247a8d 100644
--- a/src/proto_http.c
+++ b/src/proto_http.c
@@ -3189,15 +3189,15 @@ int http_handle_stats(struct stream *s, struct channel 
*req)
 /* Sets the TOS header in IPv4 and the traffic class header in IPv6 packets
  * (as per RFC3260 #4 and BCP37 #4.2 and #5.2).
  */
-void inet_set_tos(int fd, struct sockaddr_storage from, int tos)
+void inet_set_tos(int fd, const struct sockaddr_storage *from, int tos)
 {
 #ifdef IP_TOS
-   if (from.ss_family == AF_INET)
+   if (from->ss_family == AF_INET)
setsockopt(fd, IPPROTO_IP, IP_TOS, , sizeof(tos));
 #endif
 #ifdef IPV6_TCLASS
-   if (from.ss_family == AF_INET6) {
-   if (IN6_IS_ADDR_V4MAPPED(&((struct sockaddr_in6 
*))->sin6_addr))
+   if (from->ss_family == AF_INET6) {
+   if (IN6_IS_ADDR_V4MAPPED(&((struct sockaddr_in6 
*)from)->sin6_addr))
/* v4-mapped addresses need IP_TOS */
setsockopt(fd, IPPROTO_IP, IP_TOS, , sizeof(tos));
else
@@ -3363,7 +3363,7 @@ resume_execution:
 
 

Re: Crash with kernel error

[Willy Tarreau]

On Wed, May 18, 2016 at 08:58:25AM -0500, Sasha Litvak wrote:
> I have a core now.  How and what can I send to you?

Excellent :-)

If it's less than a few megs, just compress it and send it to me/us over
e-mail. In all cases, don't send it to the list as it contains some sensitive
information! If it's really large, we'll have to find some file sharing
service that doesn't expose your data to the rest of the internet. I'll
also need the unstripped haproxy binary that produced this core, and I
may even ask you later for a few libs if the stack dump is incomplete or
corrupted.

Thank you very much.

Willy

Re: Crash with kernel error

[Sasha Litvak]

I have a core now.  How and what can I send to you?

On Wed, May 18, 2016 at 8:54 AM, Willy Tarreau  wrote:

> On Wed, May 18, 2016 at 08:48:05AM -0500, Sasha Litvak wrote:
> > Home directory for user  haproxy or other permissible place had to be
> used
> > to start in order to store a core file.  While running in the non daemon
> > mode killall -6 haproxy produced core.  However editing
> /etc/init.d/haproxy
> > as below had no effect, i.e no core.
>
> Probably because the process does not have write permissions in this
> directory after it does setuid(). Also, please ensure there's no chroot
> in your config when doing this.
>
> Regards,
> Willy
>

Re: 100% cpu , epoll_wait()

[Willy Tarreau]

On Wed, May 18, 2016 at 03:39:34PM +0200, Sebastian Heid wrote:
> Hi Willy,
> 
> thanks for the feedback. I'm using OBS (Open Build Service) for creating
> CentOS/RHEL RPMs, so it should be as clean as it can get.

OK thanks for checking.

> When I find some time, I'll try to reproduce it and will come back to you.

Thank you. For now I must confess I'm clueless about what could still
cause it.

Cheers,
Willy

Re: Crash with kernel error

[Willy Tarreau]

On Wed, May 18, 2016 at 08:48:05AM -0500, Sasha Litvak wrote:
> Home directory for user  haproxy or other permissible place had to be used
> to start in order to store a core file.  While running in the non daemon
> mode killall -6 haproxy produced core.  However editing /etc/init.d/haproxy
> as below had no effect, i.e no core.

Probably because the process does not have write permissions in this
directory after it does setuid(). Also, please ensure there's no chroot
in your config when doing this.

Regards,
Willy

Re: Crash with kernel error

[Sasha Litvak]

Home directory for user  haproxy or other permissible place had to be used
to start in order to store a core file.  While running in the non daemon
mode killall -6 haproxy produced core.  However editing /etc/init.d/haproxy
as below had no effect, i.e no core.

start() {
  ++ ulimit -c unlimited
  ++ cd /home/haproxy
  quiet_check
  if [ $? -ne 0 ]; then
echo "Errors found in configuration file, check it with '$BASENAME
check'."
return 1
  fi

  echo -n "Starting $BASENAME: "
  daemon $BIN -D -f $CFG $EXTRAOPTS -p $PIDFILE
  RETVAL=$?
  echo
  [ $RETVAL -eq 0 ] && touch $LOCKFILE
  return $RETVAL
}



On Wed, May 18, 2016 at 7:34 AM, Willy Tarreau  wrote:

> On Wed, May 18, 2016 at 12:50:19PM +0200, Nenad Merdanovic wrote:
> > Hey,
> >
> > On 5/18/2016 8:28 AM, Sasha Litvak wrote:
> > > It is hard to reproduce,  It took almost a week for it to crush and
> > > produced no core.  I did ulimit -c unlimited before start.  Does it
> make
> > > sense to go to back to 1.6.3 or try git source ?
> >
> > Make sure you set the fs.suid_dumpable=1 sysctl because you are not
> > running HAproxy as root. Also, after starting make sure to check
> > /proc//limits and also that your core_pattern sysctl is set
> correctly.
>
> BTW, in order to validate the setup without having to wait for one week,
> simply send a kill -ABRT to the process. If you can't find the core,
> something is still missing.
>
> Regards,
> Willy
>
>

Hardware Security Module (HSM) + Haproxy

[Mike Ghicas]

Has anyone had any luck using a Thales nshield Connect or gemalto safenet
HSM for key storage and/or SSL sessions setup with HAproxy?
Last i could find was this from last year:
https://marc.info/?l=haproxy=143766398313038=2

Thanks
Mike

Re: Using socket commands to add a new server to a backend

[Conrad Hoffmann]

On 05/18/2016 12:35 AM, Tugberk Ugurlu wrote:
> Thanks!
> What do you think about this approach: 
> http://engineeringblog.yelp.com/2015/04/true-zero-downtime-haproxy-reloads.html
> Seems a bit complicated to get it right but wonder if it's worth investing 
> into.

That really depends on your needs. As the article mentions, haproxy only
looses 0.25% of requests while being restarted *10 times a second*. For
their third test (1M reqquests) they mention that haproxy reloaded over
1500 times during the test. As the first test had "only" 20 requests,
assuming 300 restarts during that test is a fair estimate. So while
hammering haproxy with requests (that's what a benchmark does), it looses
1/300th of 0.25% of requests during a single restart, or 0.0008%. By rough
estimates from the details given in the article, this amounts to 1.6
dropped requests - if you have 67.000 req/s!

I'll leave it up to you how much effort you think is worth putting into
reducing that number.

To give you some ideas, other options are having haproxy running on
multiple hosts and load balancing between these hosts, e.g with DNS. For
restarts, shift traffic to one instance, restart the other, then vice
versa. Such a setup will save you from much more than packet loss during
restarts, e.g. hardware failure, bad deploys, etc.

Just my two cents.

Cheers,
Conrad


> Tugberk
>> Subject: Re: Using socket commands to add a new server to a backend
>> To: tugb...@outlook.com; haproxy@formilux.org
>> From: con...@soundcloud.com
>> Date: Tue, 17 May 2016 13:40:55 +0200
>>
>> On 05/17/2016 12:53 PM, Tugberk Ugurlu wrote:
>>> Thanks for getting back!
>>> Is there a place that I can influence this change or do you think that this 
>>> is something not possible cuz of some internal architectural concerns or 
>>> similar? I am trying to mimic the AWS ELB's register and deregister 
>>> features.
>>> BTW, is HAProxy open source? If so, is it on GitHub or somewhere else?
>>
>> It most certainly is. You can find links to git repositories and other
>> resources at http://www.haproxy.org/
>>
>> Note that the main repos are _not_ on Github, but if you prefer Github
>> there is an _inofficial_ clone on Github: https://github.com/haproxy/haproxy
>>
>> For discussions and feature requests this mailing list is the right place.
>> As for this specific feature, it has been requested several times before
>> and is, as far as I understand, being worked on, but not with high priority.
>>
>>> Thanks for getting back again, I do appreciate it!
>>
>> You're welcome :)
>>
>> Conrad
>>
>>
>>> Tugberk
>>>
 Subject: Re: Using socket commands to add a new server to a backend
 To: tugb...@outlook.com; haproxy@formilux.org
 From: con...@soundcloud.com
 Date: Tue, 17 May 2016 12:30:47 +0200

 Hi Tugberk,

 On 05/16/2016 05:25 PM, Tugberk Ugurlu wrote:
> Hi,
> Is it possible to register a new server under a backend? This is similar 
> to what is being asked here: 
> http://haproxy.formilux.narkive.com/1OibZABp/using-haproxy-socket-to-add-new-servers
> I cannot see the command available under the documented list of commands: 
> http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#9.2

 The response from the thread you linked to still holds true.

> I am not sure what is the best way to handle rolling out new servers and 
> taking down the old ones without any downtime through an automated way.

 The reload in haproxy is designed to be very-close-to-zero downtime, so
 depending on your use case it might just be good enough. If you do opt for
 the solution mentioned in the above thread, do keep in mind that your
 config on disk an the state in haproxy might diverge, so your deployments
 will have take that into consideration.

 Cheers,
 Conrad
 -- 
 Conrad Hoffmann
 Traffic Engineer

 SoundCloud Ltd. | Rheinsberger Str. 76/77, 10115 Berlin, Germany

 Managing Director: Alexander Ljung | Incorporated in England & Wales
 with Company No. 6343600 | Local Branch Office | AG Charlottenburg |
 HRB 110657B
>>>   
>>>
>>
>> -- 
>> Conrad Hoffmann
>> Traffic Engineer
>>
>> SoundCloud Ltd. | Rheinsberger Str. 76/77, 10115 Berlin, Germany
>>
>> Managing Director: Alexander Ljung | Incorporated in England & Wales
>> with Company No. 6343600 | Local Branch Office | AG Charlottenburg |
>> HRB 110657B
> 
> 

-- 
Conrad Hoffmann
Traffic Engineer

SoundCloud Ltd. | Rheinsberger Str. 76/77, 10115 Berlin, Germany

Managing Director: Alexander Ljung | Incorporated in England & Wales
with Company No. 6343600 | Local Branch Office | AG Charlottenburg |
HRB 110657B

Re: 100% cpu , epoll_wait()

[Sebastian Heid]

Hi Willy,

thanks for the feedback. I'm using OBS (Open Build Service) for creating 
CentOS/RHEL RPMs, so it should be as clean as it can get.

When I find some time, I'll try to reproduce it and will come back to you.

Cheers,
Sebastian
 
-Ursprüngliche Nachricht-
> Von:Willy Tarreau 
> Gesendet: Mit 18 Mai 2016 14:44
> An: Sebastian Heid 
> CC: Lukas Tribus ; HAProxy 
> Betreff: Re: 100% cpu , epoll_wait()
> 
> Hi Sebastian,
> 
> On Thu, May 12, 2016 at 09:58:22AM +0200, Sebastian Heid wrote:
> > Hi Lukas,
> > 
> > starting from around 200mbit/s in, haproxy processes (nbproc 6) are
> > hitting 100% cpu regularly (noticed up to 3 processes at the same time with
> > 100%), but recover again on its own after some time. 
> > 
> > stracing such a process yesterday showed the following:
> > epoll_wait(0, {}, 200, 0)   = 0
> > epoll_wait(0, {}, 200, 0)   = 0
> > epoll_wait(0, {}, 200, 0)   = 0
> > epoll_wait(0, {}, 200, 0)   = 0
> > epoll_wait(0, {}, 200, 0)   = 0
> >  
> > Unfortunately I can't do any more debugging in this setup. HAproxy 1.5.14 is
> > never near to 10% cpu usage with way higher bandwidth.
> 
> So far I've got good reports from people having experienced similar issues
> with recent versions, thus I'm thinking about something, are you certain
> that you did a make clean after upgrading and before rebuilding ? Sometimes
> we tend to forget it, especially after a simple "git pull". It is very
> possible that some old .o files were not properly rebuilt and still contain
> these bugs. If in doubt, you can simply keep a copy of your latest haproxy
> binary, make clean, build again and run cmp between them. It should not
> report any difference otherwise it means there was an issue (which would be
> a great news).
> 
> Cheers,
> Willy
> 
> 
> 

Re: 100% cpu , epoll_wait()

[Willy Tarreau]

Hi Sebastian,

On Thu, May 12, 2016 at 09:58:22AM +0200, Sebastian Heid wrote:
> Hi Lukas,
> 
> starting from around 200mbit/s in, haproxy processes (nbproc 6) are
> hitting 100% cpu regularly (noticed up to 3 processes at the same time with
> 100%), but recover again on its own after some time. 
> 
> stracing such a process yesterday showed the following:
> epoll_wait(0, {}, 200, 0)   = 0
> epoll_wait(0, {}, 200, 0)   = 0
> epoll_wait(0, {}, 200, 0)   = 0
> epoll_wait(0, {}, 200, 0)   = 0
> epoll_wait(0, {}, 200, 0)   = 0
>  
> Unfortunately I can't do any more debugging in this setup. HAproxy 1.5.14 is
> never near to 10% cpu usage with way higher bandwidth.

So far I've got good reports from people having experienced similar issues
with recent versions, thus I'm thinking about something, are you certain
that you did a make clean after upgrading and before rebuilding ? Sometimes
we tend to forget it, especially after a simple "git pull". It is very
possible that some old .o files were not properly rebuilt and still contain
these bugs. If in doubt, you can simply keep a copy of your latest haproxy
binary, make clean, build again and run cmp between them. It should not
report any difference otherwise it means there was an issue (which would be
a great news).

Cheers,
Willy

Re: Crash with kernel error

[Willy Tarreau]

On Wed, May 18, 2016 at 12:50:19PM +0200, Nenad Merdanovic wrote:
> Hey,
> 
> On 5/18/2016 8:28 AM, Sasha Litvak wrote:
> > It is hard to reproduce,  It took almost a week for it to crush and
> > produced no core.  I did ulimit -c unlimited before start.  Does it make
> > sense to go to back to 1.6.3 or try git source ?
> 
> Make sure you set the fs.suid_dumpable=1 sysctl because you are not
> running HAproxy as root. Also, after starting make sure to check
> /proc//limits and also that your core_pattern sysctl is set correctly.

BTW, in order to validate the setup without having to wait for one week,
simply send a kill -ABRT to the process. If you can't find the core,
something is still missing.

Regards,
Willy

Re: Haproxy 1.6.5 listens on all IPv4 addresses

[Arthur Țițeică]

În ziua de miercuri, 18 mai 2016, la 10:30:56 EEST, Willy Tarreau a scris:
> That works for me. Let's have Arthur test them to confirm the issue goes
> away for him.

I'm glad to help.

Re: Crash with kernel error

[Nenad Merdanovic]

Hey,

On 5/18/2016 8:28 AM, Sasha Litvak wrote:
> It is hard to reproduce,  It took almost a week for it to crush and
> produced no core.  I did ulimit -c unlimited before start.  Does it make
> sense to go to back to 1.6.3 or try git source ?

Make sure you set the fs.suid_dumpable=1 sysctl because you are not
running HAproxy as root. Also, after starting make sure to check
/proc//limits and also that your core_pattern sysctl is set correctly.

Regards,
Nenad

> 
> On Thu, May 12, 2016 at 2:11 AM, Lukas Tribus  > wrote:
> 
> Hi,
> 
> 
> ok, thanks.
> 
> This probably has to do with the changes regarding buffers.
> 
> 
> If this is a lab setup, my suggestion would be you don't use the
> init scripts to start haproxy, but start it manually from the
> haproxy directory (ulimit -c unlimited; ./haproxy -f configfile),
> when haproxy crashes it should generated a file named "core" in the
> haproxy directory.
> 
> Just make sure you start haproxy as root, it doesn't matter if it
> downgrades privileges to "haproxy" after the start.
> 
> 
> 
> Thanks,
> 
> Lukas
> 
> 
> Am 12.05.2016 um 02:23 schrieb Sasha Litvak:
> 
> Lukas,
> 
> 1.6.3 didn't have any crashes.  These crashes are sporadic and
> are not happening under the load, there is very little traffic
> as we are not running production yet.  The proxy starts fine and
> can run for hours with the crash.
> Where would the core be generated?  I set it up running as user
> haproxy would I have to adjust limits for that user?
> 
> Thank you for all your help,
> 
> 
> On Wed, May 11, 2016 at 4:02 PM, Lukas Tribus    >> wrote:
> 
> Hi Sasha,
> 
> 
> so the crash happens sporadically after hours of production
> traffic? Or does it crash right away after you start it?
> 
> 
> You are saying this started with 1.6.4, what was the version you
> used before and that worked fine? 1.6.3?
> 
> 
> Before starting haproxy, enable core dumping like this:
> 
> ulimit -c unlimited
> 
> 
> Confirm its unlimited (right before starting haproxy from
> this shell):
> 
> ulimit -c
> 
> 
> 
> Disabling compiler optimizations will make sure the generated
> coredump is as meaningful as possible, you can do it like this:
> 
> make clean; make CFLAGS="-O0 -g -fno-strict-aliasing
> -Wdeclaration-after-statement" TARGET=linux2628 USE_ZLIB=1
> USE_OPENSSL=1 USE_PCRE=1
> 
> 
> But be advised that there will be performance/cpu impact, so you
> better monitor it.
> 
> 
> When you have a coredump, you can provide a backtrace with gdb
> like this:
> 
> gdb  
> 
> and issuing a "bt full"
> 
> 
> 
> 
> Regards,
> 
> Lukas
> 
> 
> 
> 
> 

Re: Haproxy 1.6.5 listens on all IPv4 addresses

[Willy Tarreau]

Hi Vincent,

On Wed, May 18, 2016 at 09:58:33AM +0200, Vincent Bernat wrote:
>  ??? 15 mai 2016 09:55 +0200, Vincent Bernat  :
> 
> >> I suppose that some new features of gcc started to rely on the
> >> strict-aliasing rule without taking -fno-strict-aliasing into
> >> consideration. I didn't find anything in the bugzilla, but it's easy to
> >> miss something as there are about 500 bugs opened about aliasing.
> >>
> >> I'll open a bug about this.
> >
> > Here is the bug:
> >  https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71120
> >
> > With luck, we should get some pointers on why and how.
> 
> So, it appears this is not a bug in GCC and it is not an aliasing
> problem but the fact that the compiler doesn't have to use memcpy() to
> copy the struct, but can copy individual member, padding excluded.

Yes, this is even a security problem which painfully affects the kernel.

> And sin_addr would be in the padding, so not copied.

That makes sense then.

> A bug has been filed
> for the glibc since the bug is in the definition:
> 
>  https://sourceware.org/bugzilla/show_bug.cgi?id=20111

Great, thanks for the link and going through this. As was mentionned
there, sockaddr_storage is expected by users and in literature to be
able to carry other addresses.

> We could either use an union like discussed previously or locally fix
> this with a memcpy(). I suppose the union would be a longer term (and
> safer since we can be exhaustive about it) fix but far more invasive.

I have the same feeling, but it would make a few switch/case occurences
cleaner or at least more explicit. For the short term and backports I
guess only memcpy() will be usable, so we should probably start by this,
backport it, then work on changing all this to a union.

> I suspect there are other uses of sockaddr_storage that could lead to
> some bugs.

If there is one, there are many.

> For example, connection.c:747. ss_family will be set to 0,
> but eventual port and address can be random. However, it seems that in
> the remaining of the code, only ss_family is important. But maybe, it
> would be better to use memset() here.

Good point.

Maybe we would need to develop a few macros (or inline functions) to
manipulate sockaddr_storage. We already have a few, I don't think we
need that many. set_port, set_addr, set_family, copy_addr and I don't
know what.

> On the other hand, on proto_tcp.c:225, bind_addr is correctly zeroed out
> with memset. Same on standard.c:829.
> 
> I also note on proto_http.c:3192, a "struct sockaddr_storage from" that
> may be more safe to replace by "struct sockaddr_storage *from" since it
> is used read-only. But maybe the compiler is smart enough to not do a
> copy and therefore the bug is not triggered here.

I didn't notice this one, good catch. It should even be a const.

> I didn't review all uses, only uses of "struct sockaddr_storage
> something". I should check for "struct sockaddr_storage *something" and
> eventual use of "*something =", but I suspect that it doesn't exist
> without a "struct sockaddr_storage something_else" around.

I think that once we get rid of *all* sockaddr_storage we'll find them
all :-)

> So, quick fix is a memcpy() in cfgparse.c and in proto_http.c:3192 and
> using memset() in connection.c:747 for consistency (so that somebody
> doesn't copy this code for something more general).
> 
> Can do the patches but union would be more work.

That works for me. Let's have Arthur test them to confirm the issue goes
away for him.

Thanks!
Willy

Re: Haproxy 1.6.5 listens on all IPv4 addresses

[Vincent Bernat]

 ❦ 15 mai 2016 09:55 +0200, Vincent Bernat  :

>> I suppose that some new features of gcc started to rely on the
>> strict-aliasing rule without taking -fno-strict-aliasing into
>> consideration. I didn't find anything in the bugzilla, but it's easy to
>> miss something as there are about 500 bugs opened about aliasing.
>>
>> I'll open a bug about this.
>
> Here is the bug:
>  https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71120
>
> With luck, we should get some pointers on why and how.

So, it appears this is not a bug in GCC and it is not an aliasing
problem but the fact that the compiler doesn't have to use memcpy() to
copy the struct, but can copy individual member, padding excluded. And
sin_addr would be in the padding, so not copied. A bug has been filed
for the glibc since the bug is in the definition:

 https://sourceware.org/bugzilla/show_bug.cgi?id=20111

We could either use an union like discussed previously or locally fix
this with a memcpy(). I suppose the union would be a longer term (and
safer since we can be exhaustive about it) fix but far more invasive.

I suspect there are other uses of sockaddr_storage that could lead to
some bugs. For example, connection.c:747. ss_family will be set to 0,
but eventual port and address can be random. However, it seems that in
the remaining of the code, only ss_family is important. But maybe, it
would be better to use memset() here.

On the other hand, on proto_tcp.c:225, bind_addr is correctly zeroed out
with memset. Same on standard.c:829.

I also note on proto_http.c:3192, a "struct sockaddr_storage from" that
may be more safe to replace by "struct sockaddr_storage *from" since it
is used read-only. But maybe the compiler is smart enough to not do a
copy and therefore the bug is not triggered here.

I didn't review all uses, only uses of "struct sockaddr_storage
something". I should check for "struct sockaddr_storage *something" and
eventual use of "*something =", but I suspect that it doesn't exist
without a "struct sockaddr_storage something_else" around.

So, quick fix is a memcpy() in cfgparse.c and in proto_http.c:3192 and
using memset() in connection.c:747 for consistency (so that somebody
doesn't copy this code for something more general).

Can do the patches but union would be more work.
-- 
A light wife doth make a heavy husband.
-- Wm. Shakespeare, "The Merchant of Venice"

Re: Crash with kernel error

[Sasha Litvak]

It is hard to reproduce,  It took almost a week for it to crush and
produced no core.  I did ulimit -c unlimited before start.  Does it make
sense to go to back to 1.6.3 or try git source ?

On Thu, May 12, 2016 at 2:11 AM, Lukas Tribus  wrote:

> Hi,
>
>
> ok, thanks.
>
> This probably has to do with the changes regarding buffers.
>
>
> If this is a lab setup, my suggestion would be you don't use the init
> scripts to start haproxy, but start it manually from the haproxy directory
> (ulimit -c unlimited; ./haproxy -f configfile), when haproxy crashes it
> should generated a file named "core" in the haproxy directory.
>
> Just make sure you start haproxy as root, it doesn't matter if it
> downgrades privileges to "haproxy" after the start.
>
>
>
> Thanks,
>
> Lukas
>
>
> Am 12.05.2016 um 02:23 schrieb Sasha Litvak:
>
>> Lukas,
>>
>> 1.6.3 didn't have any crashes.  These crashes are sporadic and are not
>> happening under the load, there is very little traffic as we are not
>> running production yet.  The proxy starts fine and can run for hours with
>> the crash.
>> Where would the core be generated?  I set it up running as user haproxy
>> would I have to adjust limits for that user?
>>
>> Thank you for all your help,
>>
>>
>> On Wed, May 11, 2016 at 4:02 PM, Lukas Tribus  lu...@gmx.net>> wrote:
>>
>> Hi Sasha,
>>
>>
>> so the crash happens sporadically after hours of production
>> traffic? Or does it crash right away after you start it?
>>
>>
>> You are saying this started with 1.6.4, what was the version you
>> used before and that worked fine? 1.6.3?
>>
>>
>> Before starting haproxy, enable core dumping like this:
>>
>> ulimit -c unlimited
>>
>>
>> Confirm its unlimited (right before starting haproxy from this shell):
>>
>> ulimit -c
>>
>>
>>
>> Disabling compiler optimizations will make sure the generated
>> coredump is as meaningful as possible, you can do it like this:
>>
>> make clean; make CFLAGS="-O0 -g -fno-strict-aliasing
>> -Wdeclaration-after-statement" TARGET=linux2628 USE_ZLIB=1
>> USE_OPENSSL=1 USE_PCRE=1
>>
>>
>> But be advised that there will be performance/cpu impact, so you
>> better monitor it.
>>
>>
>> When you have a coredump, you can provide a backtrace with gdb
>> like this:
>>
>> gdb  
>>
>> and issuing a "bt full"
>>
>>
>>
>>
>> Regards,
>>
>> Lukas
>>
>>
>>
>>
>