Re: HAProxy 1.8 built with rpath'd openssl links ok; but `haproxy -vv` reports "Built with" and "Running on" conflict
On 6/22/18 5:21 PM, William Lallemand wrote: Well, unfortunately haproxy is a very portable software which compiles with a huge number of openssl and boringssl versions, Sure. So are a lot of other apps. > it's complicated to keep everything clean but any help is welcomed. Step 1 has been simply to understand the problem. particularly with tls1.3-capable openssl 1.1.1 "ComingSoon(tm)", might be worth a review What are you suggesting there ? I'm not sure of following, is there a problem with tls1.3 in haproxy? What I'm suggesting is that there's a possibility -- as per my other post, still unclear to me -- that openssl 1.1.1, with which tls1.3 support will officially 'arrive', will have tighter restrictions on use of prior versions' APIs. Will use of v<1.1.0 apis still be just deprecated? or dropped? And, in either case, how will downstream apps -- e.g., haproxy -- deal with it. Currently, apparently, haproxy doesn't deal with the legacy-free, current Openssl api, at all. Which simply causes me some pause.
Re: HAProxy 1.8 built with rpath'd openssl links ok; but `haproxy -vv` reports "Built with" and "Running on" conflict
On 6/22/18 6:14 PM, Lukas Tribus wrote:> configuration, removing old interfaces. Drop it from your openssl > configuration, and it will work fine. > >> particularly with tls1.3-capable openssl 1.1.1 "ComingSoon(tm)", might be worth a review > > Haproxy 1.8 and -dev works fine with both openssl 1.1.1 and TLS 1.3. > 1.1.1 is API compatible with 1.1.0 and there is nothing else on the > roadmap as far as I know. A different OpenSSL API (1.2) will break all > applications *anyway*, regardless whether we all remove supposedly > obsolete interfaces today. On the other hand, likely the new API does > not have all interfaces required to replace the old functionality. > That's at least how it was with 1.1.0 (with things that where actually > removed in 1.1.0). > > The --api=1.1.0 is helpful to understand what the openssl developers > currently believe will be removed one day, but they change their > opinion all the time and a new braking API change is not even > announced at this point. That's why I suggest you don't worry about it > and compile openssl without the API restriction. No OS will ship the > openssl library with such options anyway. Have to consider this, then. The point -- or at least one of them -- of DIY'ing the openssl builds is to *not* have "deprecated" (unsupported? recommended against? whatever?) dependencies ... and an app stack built on top of them. What a distro/OS ships is of little interest, or concern. I find their packaging, particularly when it comes to security apps, often unncessarily bloated, and sometimes sloppy. To me, admittedly. So far, the 'reset' of my stack -- including e.g., but not limited to, nginx, mariadb, postfix, varnish, php, radicale, etc etc -- has no problems with the api=1.1.0 (no deprecated) openssl. Each of those projects has decided to accept/address the reality of current openssl release build options; specifically, the option to restrict use of deprecated apis. The 1st 'issue' I've hit is, apparently, haproxy. I'll have to decide whether I'm more interested in haproxy, or a consistently 'modern/current' openssl api. Atm, I'm leaning to sticking with the openssl api restriction(s). I'll need to dig further into OpenSSL 1.1.1's API ... sure, it's 1.1.0 compatible, but simply don't yet understand what the implications, if any, for deprecated APIs is. Ongoing support? Further deprecation notices? Dropped? Just unclear to me atm. What I'd prefer, not surprisingly, is an haproxy that'll handle the option. I understand there's change involved. Thanks.
Re: HAProxy 1.8 built with rpath'd openssl links ok; but `haproxy -vv` reports "Built with" and "Running on" conflict
Hello, right, your (second) build issue is caused by the --api=1.1.0 configuration, removing old interfaces. Drop it from your openssl configuration, and it will work fine. > particularly with tls1.3-capable openssl 1.1.1 "ComingSoon(tm)", might be > worth a review Haproxy 1.8 and -dev works fine with both openssl 1.1.1 and TLS 1.3. 1.1.1 is API compatible with 1.1.0 and there is nothing else on the roadmap as far as I know. A different OpenSSL API (1.2) will break all applications *anyway*, regardless whether we all remove supposedly obsolete interfaces today. On the other hand, likely the new API does not have all interfaces required to replace the old functionality. That's at least how it was with 1.1.0 (with things that where actually removed in 1.1.0). The --api=1.1.0 is helpful to understand what the openssl developers currently believe will be removed one day, but they change their opinion all the time and a new braking API change is not even announced at this point. That's why I suggest you don't worry about it and compile openssl without the API restriction. No OS will ship the openssl library with such options anyway. cheers, lukas
Re: HAProxy 1.8 built with rpath'd openssl links ok; but `haproxy -vv` reports "Built with" and "Running on" conflict
On Fri, Jun 22, 2018 at 05:04:39PM -0700, PGNet Dev wrote: > fyi, > > all the ssl 'magic' for haproxy appears to be in > > src/ssl_sock.c > > which references > >*Acknowledgement: >* We'd like to specially thank the Stud project authors for a very > clean >* and well documented code which helped us understand how the > OpenSSL API >* ought to be used in non-blocking mode. This is one difficult part > which >* is not easy to get from the OpenSSL doc, and reading the Stud code > made >* it much more obvious than the examples in the OpenSSL package. > Keep up >* the good works, guys ! >* >* Stud is an extremely efficient and scalable SSL/TLS proxy which > combines >* particularly well with haproxy. For more info about this project, > visit : >* https://github.com/bumptech/stud > > stud's not been updated in years, and per > > https://github.com/bumptech/stud > > Stud is now officially abandonware, thanks for playing. > Recommended alternative: https://github.com/varnish/hitch > > Stud's exemplar usage of the OpenSSL api is likely not the best reference for > modern openssl api usage. > This was written a long time ago during the first implementation of ssl in 2012. Before working on the SSL in HAProxy, some of us contributes to stud, but that was 6 years ago, you can easily imagine that the code evolved since then. It's not like we are using stud within haproxy. > Taking a look, instead, at the usage approach taken by recommended 'hitch', > > git clone https://github.com/varnish/hitch.git > cd hitch > > their much-simpler, openssl 1.1.0-ready implementation code is in, > > ./src/hssl_locks.c > > which notes, correctly > > /* >* OpenSSL 1.1 has a new threading implementation that no longer >* requires the application to set its own locking callbacks. >*/ > > and avoids reference to, and use of, the previously mentioned deprecated > symbols (cref: https://www.openssl.org/news/openssl-1.1.0-notes.html) > > it builds/installs > > ./bootstrap > ./configure \ > --prefix=/usr/local/hitch \ > SSL_CFLAGS="-I/usr/local/openssl/include" \ > SSL_LIBS="-L/usr/local/openssl11/lib64 > -Wl,-rpath,/usr/local/openssl11/lib64 -lssl" \ > CRYPTO_CFLAGS="-I/usr/local/openssl/include" \ > CRYPTO_LIBS="-L/usr/local/openssl11/lib64 > -Wl,-rpath,/usr/local/openssl11/lib64 -lcrypto" > make -j4 > make install > > with no errors, > > ldd /usr/local/hitch/sbin/hitch | egrep "ssl|crypto" > libssl.so.1.1 => /usr/local/openssl11/lib64/libssl.so.1.1 > (0x7f8c27cb6000) > libcrypto.so.1.1 => /usr/local/openssl11/lib64/libcrypto.so.1.1 > (0x7f8c2780d000) > > /usr/local/hitch/sbin/hitch --version > hitch 1.4.8 > Well, unfortunately haproxy is a very portable software which compiles with a huge number of openssl and boringssl versions, it's complicated to keep everything clean but any help is welcomed. > particularly with tls1.3-capable openssl 1.1.1 "ComingSoon(tm)", might be > worth a review > What are you suggesting there ? I'm not sure of following, is there a problem with tls1.3 in haproxy? -- William Lallemand
Re: HAProxy 1.8 built with rpath'd openssl links ok; but `haproxy -vv` reports "Built with" and "Running on" conflict
fyi, all the ssl 'magic' for haproxy appears to be in src/ssl_sock.c which references *Acknowledgement: * We'd like to specially thank the Stud project authors for a very clean * and well documented code which helped us understand how the OpenSSL API * ought to be used in non-blocking mode. This is one difficult part which * is not easy to get from the OpenSSL doc, and reading the Stud code made * it much more obvious than the examples in the OpenSSL package. Keep up * the good works, guys ! * * Stud is an extremely efficient and scalable SSL/TLS proxy which combines * particularly well with haproxy. For more info about this project, visit : * https://github.com/bumptech/stud stud's not been updated in years, and per https://github.com/bumptech/stud Stud is now officially abandonware, thanks for playing. Recommended alternative: https://github.com/varnish/hitch Stud's exemplar usage of the OpenSSL api is likely not the best reference for modern openssl api usage. Taking a look, instead, at the usage approach taken by recommended 'hitch', git clone https://github.com/varnish/hitch.git cd hitch their much-simpler, openssl 1.1.0-ready implementation code is in, ./src/hssl_locks.c which notes, correctly /* * OpenSSL 1.1 has a new threading implementation that no longer * requires the application to set its own locking callbacks. */ and avoids reference to, and use of, the previously mentioned deprecated symbols (cref: https://www.openssl.org/news/openssl-1.1.0-notes.html) it builds/installs ./bootstrap ./configure \ --prefix=/usr/local/hitch \ SSL_CFLAGS="-I/usr/local/openssl/include" \ SSL_LIBS="-L/usr/local/openssl11/lib64 -Wl,-rpath,/usr/local/openssl11/lib64 -lssl" \ CRYPTO_CFLAGS="-I/usr/local/openssl/include" \ CRYPTO_LIBS="-L/usr/local/openssl11/lib64 -Wl,-rpath,/usr/local/openssl11/lib64 -lcrypto" make -j4 make install with no errors, ldd /usr/local/hitch/sbin/hitch | egrep "ssl|crypto" libssl.so.1.1 => /usr/local/openssl11/lib64/libssl.so.1.1 (0x7f8c27cb6000) libcrypto.so.1.1 => /usr/local/openssl11/lib64/libcrypto.so.1.1 (0x7f8c2780d000) /usr/local/hitch/sbin/hitch --version hitch 1.4.8 particularly with tls1.3-capable openssl 1.1.1 "ComingSoon(tm)", might be worth a review
Re: HAProxy 1.8 built with rpath'd openssl links ok; but `haproxy -vv` reports "Built with" and "Running on" conflict
On 6/22/18 3:55 PM, Lukas Tribus wrote: > Hello, > > > > On Fri, 22 Jun 2018 at 22:09, PGNet Dev wrote: >>> - share the openssl config line and installation commands >> >> gcc --version >> gcc (SUSE Linux) 8.1.1 20180614 [gcc-8-branch revision 261584] >> which openssl >> /usr/local/openssl11/bin/openssl >> openssl version >> OpenSSL 1.1.0h 27 Mar 2018 >> openssl version -f > > I meant how did you install openssl in /usr/local/openssl11? What > exact config, configure and make commands? `openssl -f` provides the effective, relevant compile flags as built/installed by cd openssl-1.1.0h unset SHARED_LDFLAGS LDFLAGS LIBDEPS ZLIB_INCLUDE C_INCLUDE_PATH echo $CFLAGS -O3 -Wall -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fmessage-length=0 -grecord-gcc-switches -march=native -mtune=native make clean ./config \ --prefix=/usr/local/openssl11 \ --openssldir=/usr/local/openssl11 \ --libdir=lib64 \ --api=1.1.0 \ threads shared \ -Wl,-rpath=/usr/local/openssl11/lib64 -Wa,--noexecstack -Wl,-z,relro,-z,now -Wall -fno-common \ enable-ec_nistp_64_gcc_128 enable-rfc3779 enable-ecdsa \ no-comp no-zlib no-zlib-dynamic no-sctp no-idea no-mdc2 no-rc2 no-rc5 no-ssl3 no-weak-ssl-ciphers \ -DOPENSSL_NO_BUF_FREELISTS -DOPENSSL_NO_HEARTBEAT -DSSL_FORBID_ENULL -D_GNU_SOURCE -DPURIFY -DTERMIO make -j4 make install Works, as built above, with numerous other app builds/installs; as do a number of leaner, more restrictive builds. > Anyway I tried it myself, this is how I was successful: > > - openssl: config openssl with something like: make clean; ./config > --prefix=/home/lukas/libsslbuildpgnet/ > -Wl,-rpath=/home/lukas/libsslbuildpgnet/lib > - openssl: make && make install_sw > - haproxy: use SSL_INC and SSL_LIB properly (don't prefix it with -I > and -L), and append the rpath configuration to SSL_LIB, so in my case > that would be SSL_INC=/home/lukas/libsslbuildpgnet/include/ > SSL_LIB="/home/lukas/libsslbuildpgnet/lib > -Wl,-rpath,/home/lukas/libsslbuildpgnet/lib" Here, that still fails with the deprecated symbol checks. The build's clearly not picking up on the fact that openssl 1.1.0 is built with current api and no deprecated symbol usage. Can you provide _your_ built openssl's `openssl -f` output?
Re: HAProxy 1.8 built with rpath'd openssl links ok; but `haproxy -vv` reports "Built with" and "Running on" conflict
Hello, On Fri, 22 Jun 2018 at 22:09, PGNet Dev wrote: > > - share the openssl config line and installation commands > > gcc --version > gcc (SUSE Linux) 8.1.1 20180614 [gcc-8-branch revision 261584] > which openssl > /usr/local/openssl11/bin/openssl > openssl version > OpenSSL 1.1.0h 27 Mar 2018 > openssl version -f I meant how did you install openssl in /usr/local/openssl11? What exact config, configure and make commands? Anyway I tried it myself, this is how I was successful: - openssl: config openssl with something like: make clean; ./config --prefix=/home/lukas/libsslbuildpgnet/ -Wl,-rpath=/home/lukas/libsslbuildpgnet/lib - openssl: make && make install_sw - haproxy: use SSL_INC and SSL_LIB properly (don't prefix it with -I and -L), and append the rpath configuration to SSL_LIB, so in my case that would be SSL_INC=/home/lukas/libsslbuildpgnet/include/ SSL_LIB="/home/lukas/libsslbuildpgnet/lib -Wl,-rpath,/home/lukas/libsslbuildpgnet/lib" With that, it works for me: lukas@dev:~/haproxy-1.8$ ./haproxy -vv | grep -e HA -e "OpenSSL version" HA-Proxy version 1.8.9 2018/05/18 Built with OpenSSL version : OpenSSL 1.1.0h 27 Mar 2018 Running on OpenSSL version : OpenSSL 1.1.0h 27 Mar 2018 lukas@dev:~/haproxy-1.8$ ldd haproxy | grep -e ssl -e crypto libssl.so.1.1 => /home/lukas/libsslbuildpgnet/lib/libssl.so.1.1 (0x7f1621865000) libcrypto.so.1.1 => /home/lukas/libsslbuildpgnet/lib/libcrypto.so.1.1 (0x7f16213d6000) lukas@dev:~/haproxy-1.8$ In your initial attempt where you see 2 different version (one fips, one not) there is an build issue. You probably already have openssl 1.1 in your default directories, so haproxy finds that one and somehow compiles, but that's dangerous. cheers, lukas
Haproxy client ip
Hi I am setting up haproxy with keepalived and i need to know if is possible pass client ip for destination log server using haproxy in tcp mode? Thanks Best regards Daniel
Re: HAProxy 1.8 built with rpath'd openssl links ok; but `haproxy -vv` reports "Built with" and "Running on" conflict
On 6/22/18 12:21 PM, Lukas Tribus wrote:
> - share:
> grep OPENSSL_VERSION_TEXT /usr/include/openssl/opensslv.h
> grep OPENSSL_VERSION_TEXT /usr/local/include/openssl/opensslv.h
> grep OPENSSL_VERSION_TEXT /usr/local/openssl11/include/openssl/opensslv.h
grep OPENSSL_VERSION_TEXT /usr/include/openssl/opensslv.h
# define OPENSSL_VERSION_TEXT"OpenSSL 1.1.0h-fips 27 Mar 2018"
# define OPENSSL_VERSION_TEXT"OpenSSL 1.1.0h 27 Mar 2018"
grep OPENSSL_VERSION_TEXT /usr/local/include/openssl/opensslv.h
grep: /usr/local/include/openssl/opensslv.h: No such file or directory
grep OPENSSL_VERSION_TEXT /usr/local/openssl11/include/openssl/opensslv.h
# define OPENSSL_VERSION_TEXT"OpenSSL 1.1.0h-fips 27 Mar 2018"
# define OPENSSL_VERSION_TEXT"OpenSSL 1.1.0h 27 Mar 2018"
where
cat /usr/local/openssl11/include/openssl/opensslv.h
...
42 # define OPENSSL_VERSION_NUMBER 0x1010008fL
# ifdef OPENSSL_FIPS
# define OPENSSL_VERSION_TEXT"OpenSSL 1.1.0h-fips 27 Mar
2018"
# else
# define OPENSSL_VERSION_TEXT"OpenSSL 1.1.0h 27 Mar 2018"
# endif
...
> - don't build with PCRE for now
> - retry the SSL_INC/SSL_LIB approach (but without PCRE at all)
> - share the gcc lines of your original attempt (atleast one compile
> line and the linking)
unset LDFLAGS
make clean
make V=1 \
TARGET=linux2628 \
USE_SYSTEMD=1 \
USE_OPENSSL=1 \
ADDINC=" -I/usr/local/openssl11/include" \
ADDLIB=" -L/usr/local/openssl11/lib64 -Wl,-rpath,/usr/local/openssl11/lib64" \
ADDLIB="-ldl"
...
gcc -Iinclude -Iebtree -Wall -O2 -g -fno-strict-aliasing
-Wdeclaration-after-statement -fwrapv -fno-strict-overflow
-Wno-format-truncation -Wno-null-dereference -Wno-unused-label
-DCONFIG_HAP_LINUX_SPLICE -DTPROXY -DCONFIG_HAP_LINUX_TPROXY -DCONFIG_HAP_CRYPT
-DENABLE_POLL -DENABLE_EPOLL -DUSE_CPU_AFFINITY -DASSUME_SPLICE_WORKS
-DUSE_ACCEPT4 -DNETFILTER -DUSE_THREAD -DUSE_OPENSSL -DUSE_SYSCALL_FUTEX
-DUSE_SYSTEMD -I/usr/local/openssl11/include
-DCONFIG_HAPROXY_VERSION=\"1.8.10-ec17d7a\"
-DCONFIG_HAPROXY_DATE=\"2018/06/22\" -c -o src/ssl_sock.o src/ssl_sock.c
src/ssl_sock.c: In function ‘ssl_locking_function’:
src/ssl_sock.c:220:13: error: ‘CRYPTO_LOCK’ undeclared (first use in
this function); did you mean ‘CRYPTO_RWLOCK’?
if (mode & CRYPTO_LOCK) {
^~~
CRYPTO_RWLOCK
src/ssl_sock.c:220:13: note: each undeclared identifier is reported
only once for each function it appears in
src/ssl_sock.c:221:14: error: ‘CRYPTO_READ’ undeclared (first use in
this function); did you mean ‘CRYPTO_ONCE’?
if (mode & CRYPTO_READ)
^~~
CRYPTO_ONCE
src/ssl_sock.c: In function ‘ssl_locking_init’:
src/ssl_sock.c:238:43: warning: implicit declaration of function
‘CRYPTO_num_locks’; did you mean ‘CRYPTO_realloc’?
[-Wimplicit-function-declaration]
ssl_rwlocks = malloc(sizeof(HA_RWLOCK_T)*CRYPTO_num_locks());
^~~~
CRYPTO_realloc
src/ssl_sock.c:245:2: warning: implicit declaration of function
‘CRYPTO_set_id_callback’; did you mean ‘BIO_set_callback’?
[-Wimplicit-function-declaration]
CRYPTO_set_id_callback(ssl_id_function);
^~
BIO_set_callback
src/ssl_sock.c:246:2: warning: implicit declaration of function
‘CRYPTO_set_locking_callback’; did you mean ‘BIO_set_info_callback’?
[-Wimplicit-function-declaration]
CRYPTO_set_locking_callback(ssl_locking_function);
^~~
BIO_set_info_callback
src/ssl_sock.c: In function ‘ssl_sock_do_create_cert’:
src/ssl_sock.c:1693:23: warning: implicit declaration of function
‘X509_get_notBefore’; did you mean ‘X509_getm_notBefore’?
[-Wimplicit-function-declaration]
if (!X509_gmtime_adj(X509_get_notBefore(newcrt), (long)-60*60*24) ||
^~
X509_getm_notBefore
src/ssl_sock.c:1693:23: warning: passing argument 1 of
‘X509_gmtime_adj’ makes pointer from integer without a cast [-Wint-conversion]
if (!X509_gmtime_adj(X509_get_notBefore(newcrt), (long)-60*60*24) ||
^~
In file included from /usr/local/openssl11/include/openssl/pem.h:17,
from /usr/local/openssl11/include/openssl/ssl.h:55,
from src/ssl_sock.c:43:
/usr/local/openssl11/include/openssl/x509.h:479:12: note: expected
‘ASN1_TIME *’ {aka ‘struct asn1_string_st *’} but argument is of type ‘int’
ASN1_TIME
Re: [PATCH] MINOR: mworker: exit with 0 on successful exit
❦ 22 juin 2018 22:03 +0200, Vincent Bernat :
> if (current_child(exitpid)) {
> ha_alert("Current worker %d exited with code
> %d\n", exitpid, status);
This is a lie, but I don't think it matters much. We could (mentally)
translate "with code 0" by "with success".
--
Choose a data representation that makes the program simple.
- The Elements of Programming Style (Kernighan & Plauger)
[PATCH] MINOR: mworker: exit with 0 on successful exit
Without this patch, when killing the master process, the SIGTERM
signal is forwarded to all children. Last children will likely exit
with "killed by signal SIGTERM" status which would be converted by an
exit with status 143 of the master process.
With this patch, the master process takes note it is requesting its
children to stop and will convert "killed by signal SIGTERM" to an
exit status of 0. Therefore, the master process will exit with status
0 if everything happens as expected.
Killing a worker process with SIGTERM will still trigger an exit of
the master process with a status of 143.
---
src/haproxy.c | 13 ++---
1 file changed, 10 insertions(+), 3 deletions(-)
diff --git a/src/haproxy.c b/src/haproxy.c
index 11d1d47ceb41..2f11ad124873 100644
--- a/src/haproxy.c
+++ b/src/haproxy.c
@@ -723,6 +723,7 @@ static void mworker_wait()
{
int exitpid = -1;
int status = 0;
+ int exiting = 0;
restart_wait:
@@ -749,6 +750,7 @@ restart_wait:
}
#endif
ha_warning("Exiting Master process...\n");
+ exiting = 1;
mworker_kill(sig);
mworker_unregister_signals();
}
@@ -763,8 +765,13 @@ restart_wait:
if (WIFEXITED(status))
status = WEXITSTATUS(status);
- else if (WIFSIGNALED(status))
- status = 128 + WTERMSIG(status);
+ else if (WIFSIGNALED(status)) {
+ if (exiting && (WTERMSIG(status) == SIGINT ||
+ WTERMSIG(status) == SIGTERM))
+ status = 0;
+ else
+ status = 128 + WTERMSIG(status);
+ }
else if (WIFSTOPPED(status))
status = 128 + WSTOPSIG(status);
else
@@ -776,7 +783,7 @@ restart_wait:
/* check if exited child was in the current children
list */
if (current_child(exitpid)) {
ha_alert("Current worker %d exited with code
%d\n", exitpid, status);
- if (status != 0 && status != 130 && status !=
143
+ if (status != 0
&& !(global.tune.options &
GTUNE_NOEXIT_ONFAILURE)) {
ha_alert("exit-on-failure: killing
every workers with SIGTERM\n");
mworker_kill(SIGTERM);
--
2.18.0
Re: [PATCH] MINOR: systemd: consider exit status 143 as successful
On Fri, Jun 22, 2018 at 08:57:03PM +0200, Vincent Bernat wrote: > The master process will exit with the status of the last worker. When > the worker is killed with SIGTERM, it is expected to get 143 as an > exit status. Therefore, we consider this exit status as normal from a > systemd point of view. If it happens when not stopping, the systemd > unit is configured to always restart, so it has no adverse effect. > > This has mostly a cosmetic effect. Without the patch, stopping HAProxy > leads to the following status: > > ● haproxy.service - HAProxy Load Balancer >Loaded: loaded (/lib/systemd/system/haproxy.service; disabled; vendor > preset: enabled) >Active: failed (Result: exit-code) since Fri 2018-06-22 20:35:42 CEST; > 8min ago > Docs: man:haproxy(1) >file:/usr/share/doc/haproxy/configuration.txt.gz > Process: 32715 ExecStart=/usr/sbin/haproxy -Ws -f $CONFIG -p $PIDFILE > $EXTRAOPTS (code=exited, status=143) > Process: 32714 ExecStartPre=/usr/sbin/haproxy -f $CONFIG -c -q > $EXTRAOPTS (code=exited, status=0/SUCCESS) > Main PID: 32715 (code=exited, status=143) > > After the patch: > > ● haproxy.service - HAProxy Load Balancer >Loaded: loaded (/lib/systemd/system/haproxy.service; disabled; vendor > preset: enabled) >Active: inactive (dead) > Docs: man:haproxy(1) >file:/usr/share/doc/haproxy/configuration.txt.gz > --- > contrib/systemd/haproxy.service.in | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/contrib/systemd/haproxy.service.in > b/contrib/systemd/haproxy.service.in > index 7a8b6bead2df..74e66e302065 100644 > --- a/contrib/systemd/haproxy.service.in > +++ b/contrib/systemd/haproxy.service.in > @@ -10,6 +10,7 @@ ExecReload=@SBINDIR@/haproxy -f $CONFIG -c -q > ExecReload=/bin/kill -USR2 $MAINPID > KillMode=mixed > Restart=always > +SuccessExitStatus=143 > Type=notify > > # The following lines leverage SystemD's sandboxing options to provide Hm that's interesting, but isn't it better to consider that the SIGTERM is a "normal" status code, and let the master leaves with zero, instead of changing systemd configuration? We could do the same with 130 (SIGINT). In fact those two exits codes are already handled differently in the master. -- William Lallemand
Re: HAProxy 1.8 built with rpath'd openssl links ok; but `haproxy -vv` reports "Built with" and "Running on" conflict
Hello, On Fri, 22 Jun 2018 at 20:45, PGNet Dev wrote: > with 'your' advised "actual paths", and from Makefile > > # OpenSSL is packaged in various forms and with various dependencies. > # In general -lssl is enough, but on some platforms, -lcrypto may be > needed, > # reason why it's added by default. Some even need -lz, then you'll > need to > # pass it in the "ADDLIB" variable if needed. If your SSL libraries > are not > # in the usual path, use SSL_INC=/path/to/inc and > SSL_LIB=/path/to/lib. > > build fails, referencing deprecated, pre openssl 1.1.0 symbols, What I suggested was: > make V=1 \ > TARGET=linux2628 \ > USE_SYSTEMD=1 \ > USE_OPENSSL=1 \ > ADDLIB=" -I/usr/local/openssl11/include" \ > ADDLIB=" -L/usr/local/openssl11/lib64 > -Wl,-rpath,/usr/local/openssl11/lib64" \ > ADDLIB="-ldl" Anyway please stop including PCRE while troubleshooting this build issue. It only complicates things. You can include it again when you fixed the openssl build issue. I suggest: - don't build with PCRE for now - use the ADDLIB approach like suggest above - retry the SSL_INC/SSL_LIB approach (but without PCRE at all) - share the gcc lines of your original attempt (atleast one compile line and the linking) - share the openssl config line and installation commands - share: grep OPENSSL_VERSION_TEXT /usr/include/openssl/opensslv.h grep OPENSSL_VERSION_TEXT /usr/local/include/openssl/opensslv.h grep OPENSSL_VERSION_TEXT /usr/local/openssl11/include/openssl/opensslv.h Also, if you just want to test a specific openssl version locally, you may just compile it statically, as per README: http://git.haproxy.org/?p=haproxy.git;a=blob;f=README;h=1b012cc5e9100c74c5143ff4d3b0625ca07f648f;hb=ba86c6c25bf252e44589ae2b4d51a67c4f47d244#l123 cheers, lukas
BUG: Tw is negative with lua sleep
When using core.msleep in lua, the %Tw metric is a negative value.
For example with the following config:
haproxy.cfg:
global
lua-load /tmp/haproxy.lua
frontend f1
mode http
bind :8000
default_backend b1
log 127.0.0.1:1234 daemon
log-format Ta=%Ta\ Tc=%Tc\ Td=%Td\ Th=%Th\ Ti=%Ti\ Tq=%Tq\
TR=%TR\ Tr=%Tr\ Tt=%Tt\ Tw=%Tw
backend b1
mode http
http-request use-service lua.foo
haproxy.lua:
core.register_service("foo", "http", function(applet)
core.msleep(100)
applet:set_status(200)
applet:start_response()
end)
The log contains:
Ta=104 Tc=0 Td=0 Th=0 Ti=0 Tq=104 TR=104 Tr=104 Tt=104 Tw=-104
^ TR also looks wrong, as it did not take 104ms to receive the full request.
This is built from the commit before current master: d8fd2af
-Patrick
[PATCH] MINOR: systemd: consider exit status 143 as successful
The master process will exit with the status of the last worker. When the worker is killed with SIGTERM, it is expected to get 143 as an exit status. Therefore, we consider this exit status as normal from a systemd point of view. If it happens when not stopping, the systemd unit is configured to always restart, so it has no adverse effect. This has mostly a cosmetic effect. Without the patch, stopping HAProxy leads to the following status: ● haproxy.service - HAProxy Load Balancer Loaded: loaded (/lib/systemd/system/haproxy.service; disabled; vendor preset: enabled) Active: failed (Result: exit-code) since Fri 2018-06-22 20:35:42 CEST; 8min ago Docs: man:haproxy(1) file:/usr/share/doc/haproxy/configuration.txt.gz Process: 32715 ExecStart=/usr/sbin/haproxy -Ws -f $CONFIG -p $PIDFILE $EXTRAOPTS (code=exited, status=143) Process: 32714 ExecStartPre=/usr/sbin/haproxy -f $CONFIG -c -q $EXTRAOPTS (code=exited, status=0/SUCCESS) Main PID: 32715 (code=exited, status=143) After the patch: ● haproxy.service - HAProxy Load Balancer Loaded: loaded (/lib/systemd/system/haproxy.service; disabled; vendor preset: enabled) Active: inactive (dead) Docs: man:haproxy(1) file:/usr/share/doc/haproxy/configuration.txt.gz --- contrib/systemd/haproxy.service.in | 1 + 1 file changed, 1 insertion(+) diff --git a/contrib/systemd/haproxy.service.in b/contrib/systemd/haproxy.service.in index 7a8b6bead2df..74e66e302065 100644 --- a/contrib/systemd/haproxy.service.in +++ b/contrib/systemd/haproxy.service.in @@ -10,6 +10,7 @@ ExecReload=@SBINDIR@/haproxy -f $CONFIG -c -q ExecReload=/bin/kill -USR2 $MAINPID KillMode=mixed Restart=always +SuccessExitStatus=143 Type=notify # The following lines leverage SystemD's sandboxing options to provide -- 2.18.0
Re: HAProxy 1.8 built with rpath'd openssl links ok; but `haproxy -vv` reports "Built with" and "Running on" conflict
Hi,
On 6/22/18 10:57 AM, Lukas Tribus wrote:
> SSL_INC and SSL_LIB expect actual paths, not additional commands.
> Replaces both with ADDLIB. Also you don't need to specify -lssl
> -lcrypt, USE_OPENSSL does not for you.
Then a bit confused by what I'm seeing atm.
With 'my' current, if incorrect, flags
unset LDFLAGS
make clean
make V=1 -j4 \
TARGET=linux2628 \
USE_PCRE2=1 \
USE_PCRE2_JIT=1 \
USE_OPENSSL=1 \
SSL_INC=" -I/usr/local/openssl11/include" \
SSL_LIB=" -L/usr/local/openssl11/lib64
-Wl,-rpath,/usr/local/openssl11/lib64" \
PCRE2_INC=" -I/usr/local/include" \
PCRE2_LIB=" -L/usr/local/lib64 -Wl,-rpath,/usr/local/lib64"
./haproxy -v
HA-Proxy version 1.8.10-ec17d7a 2018/06/22
ldd ./haproxy | egrep "ssl|crypto|pcre"
libssl.so.1.1 => /usr/local/openssl11/lib64/libssl.so.1.1
(0x7f8debd8a000)
libcrypto.so.1.1 => /usr/local/openssl11/lib64/libcrypto.so.1.1
(0x7f8deb8ef000)
libpcre2-8.so.0 => /usr/local/lib64/libpcre2-8.so.0
(0x7f8deb666000)
libpcre2-posix.so.2 => /usr/local/lib64/libpcre2-posix.so.2
(0x7f8deb463000)
with 'your' advised "actual paths", and from Makefile
# OpenSSL is packaged in various forms and with various dependencies.
# In general -lssl is enough, but on some platforms, -lcrypto may be
needed,
# reason why it's added by default. Some even need -lz, then you'll
need to
# pass it in the "ADDLIB" variable if needed. If your SSL libraries are
not
# in the usual path, use SSL_INC=/path/to/inc and SSL_LIB=/path/to/lib.
build fails, referencing deprecated, pre openssl 1.1.0 symbols,
unset LDFLAGS
make clean
make V=1 -j4 \
TARGET=linux2628 \
USE_PCRE2=1 \
USE_PCRE2_JIT=1 \
USE_OPENSSL=1 \
SSL_INC="/usr/local/openssl11/include" \
SSL_LIB="/usr/local/openssl11/lib64" \
PCRE2_INC="/usr/local/include" \
PCRE2_LIB="/usr/local/lib64"
...
gcc -Iinclude -Iebtree -Wall -O2 -g -fno-strict-aliasing
-Wdeclaration-after-statement -fwrapv -fno-strict-overflow
-Wno-format-truncation -Wno-null-dereference -Wno-unused-label
-DCONFIG_HAP_LINUX_SPLICE -DTPROXY -DCONFIG_HAP_LINUX_TPROXY -DCONFIG_HAP_CRYPT
-DENABLE_POLL -DENABLE_EPOLL -DUSE_CPU_AFFINITY -DASSUME_SPLICE_WORKS
-DUSE_ACCEPT4 -DNETFILTER -DUSE_THREAD -DUSE_OPENSSL
-I/usr/local/openssl11/include -DUSE_SYSCALL_FUTEX -DUSE_PCRE2
-DPCRE2_CODE_UNIT_WIDTH=8 -I/usr/local/include -DUSE_PCRE2_JIT
-DCONFIG_HAPROXY_VERSION=\"1.8.10-ec17d7a\"
-DCONFIG_HAPROXY_DATE=\"2018/06/22\" -c -o src/ssl_sock.o src/ssl_sock.c
src/ssl_sock.c: In function ‘ssl_locking_function’:
src/ssl_sock.c:220:13: error: ‘CRYPTO_LOCK’ undeclared (first
use in this function); did you mean ‘CRYPTO_RWLOCK’?
if (mode & CRYPTO_LOCK) {
^~~
CRYPTO_RWLOCK
src/ssl_sock.c:220:13: note: each undeclared identifier is
reported only once for each function it appears in
src/ssl_sock.c:221:14: error: ‘CRYPTO_READ’ undeclared (first
use in this function); did you mean ‘CRYPTO_ONCE’?
if (mode & CRYPTO_READ)
^~~
CRYPTO_ONCE
src/ssl_sock.c: In function ‘ssl_locking_init’:
src/ssl_sock.c:238:43: warning: implicit declaration of
function ‘CRYPTO_num_locks’; did you mean ‘CRYPTO_realloc’?
[-Wimplicit-function-declaration]
ssl_rwlocks = malloc(sizeof(HA_RWLOCK_T)*CRYPTO_num_locks());
^~~~
CRYPTO_realloc
src/ssl_sock.c:245:2: warning: implicit declaration of function
‘CRYPTO_set_id_callback’; did you mean ‘BIO_set_callback’?
[-Wimplicit-function-declaration]
CRYPTO_set_id_callback(ssl_id_function);
^~
BIO_set_callback
src/ssl_sock.c:246:2: warning: implicit declaration of function
‘CRYPTO_set_locking_callback’; did you mean ‘BIO_set_info_callback’?
[-Wimplicit-function-declaration]
CRYPTO_set_locking_callback(ssl_locking_function);
^~~
BIO_set_info_callback
src/ssl_sock.c: In function ‘ssl_sock_do_create_cert’:
src/ssl_sock.c:1693:23: warning: implicit declaration of
function ‘X509_get_notBefore’; did you mean ‘X509_getm_notBefore’?
[-Wimplicit-function-declaration]
if (!X509_gmtime_adj(X509_get_notBefore(newcrt),
(long)-60*60*24) ||
BUG: cannot take the address of an rvalue of type 'unsigned long'
Trying to compile current master on OS-X, and get:
gcc -Iinclude -Iebtree -Wall -O0 -g -fno-strict-aliasing
-Wdeclaration-after-statement -fwrapv -fno-strict-overflow
-Wno-address-of-packed-member -Wno-null-dereference
-Wno-unused-label -DTPROXY -DUSE_ZLIB -DENABLE_POLL
-DENABLE_KQUEUE -DUSE_OPENSSL
-I/Users/phemmer/git/haproxy/openssl-1.1.0h/include -DUSE_LUA
-I/Users/phemmer/git/haproxy/lua-5.3.4/src
-I/Users/phemmer/git/haproxy/lua-5.3.4/src -DUSE_PCRE
-I/opt/local/include -DCONFIG_HAPROXY_VERSION=\"1.9-dev0-ba86c6-462\"
-DCONFIG_HAPROXY_DATE=\"2018/06/22\" \
-DBUILD_TARGET='"osx"' \
-DBUILD_ARCH='""' \
-DBUILD_CPU='"generic"' \
-DBUILD_CC='"gcc"' \
-DBUILD_CFLAGS='"-O0 -g -fno-strict-aliasing
-Wdeclaration-after-statement -fwrapv -fno-strict-overflow
-Wno-address-of-packed-member -Wno-null-dereference -Wno-unused-label"' \
-DBUILD_OPTIONS='"USE_ZLIB=1 USE_OPENSSL=1
USE_LUA=1 USE_PCRE=1"' \
-c -o src/haproxy.o src/haproxy.c
clang: warning: argument unused during compilation:
'-fno-strict-overflow' [-Wunused-command-line-argument]
src/haproxy.c:2476:16: error: cannot take the address of an
rvalue of type 'unsigned long'
HA_ATOMIC_AND(_threads_mask, ~tid_bit);
^
include/common/hathreads.h:41:42: note: expanded from macro
'HA_ATOMIC_AND'
#define HA_ATOMIC_AND(val, flags)({*(val) &= (flags);})
^~~
1 error generated.
make: *** [src/haproxy.o] Error 1
Built with: Apple LLVM version 9.0.0 (clang-900.0.39.2)
This broke in change ba86c6c:
commit ba86c6c25bf252e44589ae2b4d51a67c4f47d244 (HEAD -> master,
origin/master, origin/HEAD)
Author: Christopher Faulet
Date: Thu Jun 21 09:57:39 2018 +0200
MINOR: threads: Be sure to remove threads from all_threads_mask on exit
When HAProxy is started with several threads, Each running thread
holds a bit in
the bitfiled all_threads_mask. This bitfield is used here and there
to check
which threads are registered to take part in a specific processing.
So when a
thread exits, it seems normal to remove it from all_threads_mask.
No direct impact could be identified with this right now but it would
be better to backport it to 1.8 as a preventive measure to avoid complex
situations like the one in previous bug.
-Patrick
Re: HAProxy 1.8 built with rpath'd openssl links ok; but `haproxy -vv` reports "Built with" and "Running on" conflict
Hello, > make V=1 \ > TARGET=linux2628 \ > USE_SYSTEMD=1 \ > USE_OPENSSL=1 \ > SSL_INC=" -I/usr/local/openssl11/include" \ > SSL_LIB=" -L/usr/local/openssl11/lib64 > -Wl,-rpath,/usr/local/openssl11/lib64" \ > ADDLIB="-ldl -lssl -lcrypto" SSL_INC and SSL_LIB expect actual paths, not additional commands. Replaces both with ADDLIB. Also you don't need to specify -lssl -lcrypt, USE_OPENSSL does not for you. Try: > make V=1 \ > TARGET=linux2628 \ > USE_SYSTEMD=1 \ > USE_OPENSSL=1 \ > ADDLIB=" -I/usr/local/openssl11/include" \ > ADDLIB=" -L/usr/local/openssl11/lib64 > -Wl,-rpath,/usr/local/openssl11/lib64" \ > ADDLIB="-ldl" You may not even need -ldl. Lukas
Stats page access over SSL fails at "Error code: SSL_ERROR_NO_CYPHER_OVERLAP"; no-SSL, ok
I've enabled stats on haproxy 1.8.10 With haproxy.conf ... listen stats ... bind 192.0.2.1:1234 ... the non-ssl stats web page is fully accessible/functional. If I enable ssl for it, 1st concatenating my crt & key cat haproxy.crt.pem haproxy.key.pem > haproxy.CONCAT.crt.pem Checking the cert, with my CA cert openssl verify \ -CAfile /usr/local/etc/haproxy/ssl/myCA.CHAIN.crt.pem \ /usr/local/etc/haproxy/ssl/haproxy.CONCAT.pem haproxy.CONCAT.crt.pem: OK Verifying cipher support openssl ciphers -tls1_2 ...:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:... then configuring haproxy.conf global ... + ssl-default-bind-ciphers ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305 + ssl-default-bind-options force-tlsv12 ... listen stats ... - bind 192.0.2.1:1234 + bind 192.0.2.1:1234 ssl crt /usr/local/etc/haproxy/ssl/haproxy.CONCAT.crt.pem ca-file /usr/local/etc/haproxy/ssl/myCA.CHAIN.crt.pem ... secure access to the stats page fails, Cannot communicate securely with peer: no common encryption algorithm(s). Error code: SSL_ERROR_NO_CYPHER_OVERLAP I regularly use my generated certs/keys with CHACHA20 ciphers elsewhere, so they're not _inherently_ the issue. If not a simple config issue on my end, I suspect it's possible this is (?) related to the 'mystery' Openssl lib linking issue I've having, @ https://www.mail-archive.com/haproxy@formilux.org/msg30448.html
Re: HAProxy 1.8 built with rpath'd openssl links ok; but `haproxy -vv` reports "Built with" and "Running on" conflict
> One possible cause could be pcre-config reporting -I/usr and causing > -I/usr/inc to be included before the SSL_INC path. fwiw, here, at my shell/env pcre-config --libs --libs-posix --libs-cpp --cflags --cflags-posix -L/usr/local/lib64 -lpcre -L/usr/local/lib64 -lpcreposix -lpcre -L/usr/local/lib64 -lpcrecpp -lpcre -I/usr/local/include -I/usr/local/include pcre2-config --libs8 --libs-posix --cflags --cflags-posix -L/usr/local/lib64 -lpcre2-8 -L/usr/local/lib64 -lpcre2-posix -lpcre2-8 -I/usr/local/include -I/usr/local/include
Re: HAProxy 1.8 built with rpath'd openssl links ok; but `haproxy -vv` reports "Built with" and "Running on" conflict
> Now to track down what's sourcing that "libpcre.so.1 =>
> /usr/lib64/libpcre.so.1"
looks like it's getting pulled in by libselinux & libsystemd ...
for l in $(ldd ./haproxy | awk '{print $3}')
do
echo -e "LIB: ${l}"
ldd ${l} | grep libpcre
done
LIB: /lib64/libcrypt.so.1
LIB: /lib64/libdl.so.2
LIB: /lib64/libpthread.so.0
LIB: /usr/local/openssl11/lib64/libssl.so.1.1
LIB: /usr/local/openssl11/lib64/libcrypto.so.1.1
!!! LIB: /usr/lib64/libsystemd.so.0
!!! libpcre.so.1 => /usr/lib64/libpcre.so.1 (0x7f8ceaf4)
LIB: /usr/local/lib64/libpcre2-8.so.0
LIB: /usr/local/lib64/libpcre2-posix.so.2
libpcre2-8.so.0 => /usr/local/lib64/libpcre2-8.so.0
(0x7fc22474f000)
LIB: /lib64/libc.so.6
LIB: /lib64/libresolv.so.2
!!! LIB: /lib64/libselinux.so.1
!!! libpcre.so.1 => /usr/lib64/libpcre.so.1 (0x7f0d94f13000)
LIB: /usr/lib64/libcap.so.2
LIB: /lib64/librt.so.1
LIB: /usr/lib64/liblzma.so.5
LIB: /usr/lib64/liblz4.so.1
LIB: /usr/lib64/libgcrypt.so.20
LIB: /usr/lib64/libgpg-error.so.0
LIB: /usr/lib64/libpcre.so.1
whether or not that's causing the -vv 'artifact' or not, dunno yet
Re: HAProxy 1.8 built with rpath'd openssl links ok; but `haproxy -vv` reports "Built with" and "Running on" conflict
On 6/22/18 7:44 AM, Willy Tarreau wrote: > Did you try to disable or adjust the PCRE flags as I indicated in > previous e-mail to see if it's the side effect ? I have _now_ ... needed a few minutes. > One possible cause could be pcre-config reporting -I/usr and causing > -I/usr/inc to be included before the SSL_INC path. A quick test could consist > in building without PCRE to validate or invalidate that assumption. If it > fixes it, please retry with "PCREDIR=" so that it doesn't try to set > PCRE_INC/PCRE_LIB. There's certainly a pcre-related 'oddity' ... regardless of build with, or without, PCRE, it's linking *something* with pcre1 lib. Notice ... Built WITH pcre2's libs spec'd make V=1 \ TARGET=linux2628 \ USE_SYSTEMD=1 \ USE_PCRE2=1 \ USE_PCRE2_JIT=1 \ USE_OPENSSL=1 \ SSL_INC=" -I/usr/local/openssl11/include" \ SSL_LIB=" -L/usr/local/openssl11/lib64 -Wl,-rpath,/usr/local/openssl11/lib64" \ PCRE2_INC=" -I/usr/local/include" \ PCRE2_LIB=" -L/usr/local/lib64 -Wl,-rpath,/usr/local/lib64" \ ADDLIB="-ldl -lssl -lcrypto -lpcre2-posix -lpcre2-8" ldd ./haproxy | egrep -i "ssl|crypto|pcre" libssl.so.1.1 => /usr/local/openssl11/lib64/libssl.so.1.1 (0x7f1d575c8000) libcrypto.so.1.1 => /usr/local/openssl11/lib64/libcrypto.so.1.1 (0x7f1d5712d000) libpcre2-8.so.0 => /usr/local/lib64/libpcre2-8.so.0 (0x7f1d56c0f000) libpcre2-posix.so.2 => /usr/local/lib64/libpcre2-posix.so.2 (0x7f1d56a0c000) ??? libpcre.so.1 => /usr/lib64/libpcre.so.1 (0x7f27070bd000) haproxy -vv ... Built with OpenSSL version : OpenSSL 1.1.0h-fips 27 Mar 2018 Running on OpenSSL version : OpenSSL 1.1.0h 27 Mar 2018 ... Built withOUT pcre2 make V=1 \ TARGET=linux2628 \ USE_SYSTEMD=1 \ USE_OPENSSL=1 \ SSL_INC=" -I/usr/local/openssl11/include" \ SSL_LIB=" -L/usr/local/openssl11/lib64 -Wl,-rpath,/usr/local/openssl11/lib64" \ ADDLIB="-ldl -lssl -lcrypto" ... Built without PCRE or PCRE2 support (using libc's regex instead) ... ldd ./haproxy | egrep -i "ssl|crypto|pcre" libssl.so.1.1 => /usr/local/openssl11/lib64/libssl.so.1.1 (0x7f34c9591000) libcrypto.so.1.1 => /usr/local/openssl11/lib64/libcrypto.so.1.1 (0x7f34c90f6000) ??? libpcre.so.1 => /usr/lib64/libpcre.so.1 (0x7f0a0c9d9000) haproxy -vv Built with OpenSSL version : OpenSSL 1.1.0h-fips 27 Mar 2018 Running on OpenSSL version : OpenSSL 1.1.0h 27 Mar 2018 Now to track down what's sourcing that "libpcre.so.1 => /usr/lib64/libpcre.so.1"
Re: HAProxy 1.8 built with rpath'd openssl links ok; but `haproxy -vv` reports "Built with" and "Running on" conflict
On Fri, Jun 22, 2018 at 07:03:36AM -0700, PGNet Dev wrote: > Hi > > On 6/21/18 10:50 PM, Willy Tarreau wrote:> Well, first please ensure you're > building latest fixes and not and old > > version otherwise you'll expose yourself to all these known and now fixed > > bugs : > > oops, missed that! :-/ > > switching to 'latest' in branch, v1.8.9, still see the same in "-vv" output, > > ldd `which haproxy` | egrep "ssl|crypto" > libssl.so.1.1 => /usr/local/openssl11/lib64/libssl.so.1.1 > (0x7f6a21894000) > libcrypto.so.1.1 => /usr/local/openssl11/lib64/libcrypto.so.1.1 > (0x7f6a213f9000) > > > haproxy -vv > HA-Proxy version 1.8.9-83616ec 2018/05/18 > Copyright 2000-2018 Willy Tarreau > > Build options : > TARGET = linux2628 > CPU = generic > CC = gcc > CFLAGS = -O2 -g -fno-strict-aliasing > -Wdeclaration-after-statement -fwrapv -fno-strict-overflow > -Wno-format-truncation -Wno-null-dereference -Wno-unused-label > OPTIONS = USE_OPENSSL=1 USE_SYSTEMD=1 USE_PCRE2=1 > USE_PCRE2_JIT=1 > > Default settings : > maxconn = 2000, bufsize = 16384, maxrewrite = 1024, > maxpollevents = 200 > > Built with OpenSSL version : OpenSSL 1.1.0h-fips 27 Mar 2018 > Running on OpenSSL version : OpenSSL 1.1.0h 27 Mar 2018 > ... > > > Again, not sure if it's a 'real' problem, or just a report artifact. Did you try to disable or adjust the PCRE flags as I indicated in previous e-mail to see if it's the side effect ? Willy
[ANNOUNCE] haproxy-1.8.10
Hi,
HAProxy 1.8.10 was released on 2018/06/22. It added 40 new commits
after version 1.8.9.
This version fixes several bugs that were crashing haproxy, when using
http-request set-map with a wrong type or with the cirpherlist capture.
Some deadlocks and crashes were also fixed that were provoked by the lua.
There was also an issue with the poller which was using 100% of the CPU.
It fixes several issues where haproxy was not leaving anymore when asked for a
soft stop, or a reload in master-worker mode. These issues were amplified with
the use of nbthread.
The SPOP version has been upgraded to 2.0. Due to an inconsistency in the
examples and the reference implementation the 1.0 version is not supported
anymore. This new version is basically the 1.0 with a bugfix. If your SPOP
agent does not work anymore, you must upgrade the version and fix the SPOP
flags. See commits c4dcaff, 48d02d0, 633f3bf in the master.
Thanks to all people who reported and helped fix those issues.
Users are encouraged to update.
Please find the usual URLs below :
Site index : http://www.haproxy.org/
Discourse: http://discourse.haproxy.org/
Sources : http://www.haproxy.org/download/1.8/src/
Git repository : http://git.haproxy.org/git/haproxy-1.8.git/
Git Web browsing : http://git.haproxy.org/?p=haproxy-1.8.git
Changelog: http://www.haproxy.org/download/1.8/src/CHANGELOG
Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/
---
Complete changelog :
Christopher Faulet (9):
BUG/MEDIUM: contrib/mod_defender: Use network order to encode/decode flags
BUG/MEDIUM: contrib/modsecurity: Use network order to encode/decode flags
BUG/MEDIUM: spoe: Return an error when the wrong ACK is received in sync
mode
MAJOR: spoe: upgrade the SPOP version to 2.0 and remove the support for
1.0
BUG/MINOR: contrib/spoa_example: Don't reset the status code during
disconnect
BUG/MINOR: contrib/mod_defender: Don't reset the status code during
disconnect
BUG/MINOR: contrib/modsecurity: Don't reset the status code during
disconnect
BUG/MEDIUM: threads: Use the sync point to check active jobs and exit
MINOR: threads: Be sure to remove threads from all_threads_mask on exit
Daniel Corbett (2):
BUG/MEDIUM: servers: Add srv_addr default placeholder to the state file
BUG/MEDIUM: stick-tables: Decrement ref_cnt in table_* converters
David Carlier (1):
DOC: contrib/modsecurity: few typo fixes
Dragan Dosen (2):
BUG/MINOR: contrib/mod_defender: update pointer on the end of the frame
BUG/MINOR: contrib/modsecurity: update pointer on the end of the frame
Frédéric Lécaille (1):
BUG/MINOR: lua: Segfaults with wrong usage of types.
Kevin Zhu (1):
DOC: SPOE.txt: fix a typo
Olivier Houchard (4):
BUG/MEDIUM: dns: Delay the attempt to run a DNS resolution on check
failure.
BUG/MEDIUM: fd: Only check update_mask against all_threads_mask.
BUG/MINOR: unix: Make sure we can transfer abns sockets on seamless
reload.
BUG/MEDIUM: fd: Don't modify the update_mask in fd_dodelete().
Thierry FOURNIER (11):
BUG/MEDIUM: spoe: Flags are not encoded in network order
BUG/MEDIUM: lua/socket: Length required read doesn't work
MINOR: task/notification: Is notifications registered ?
BUG/MEDIUM: lua/socket: wrong scheduling for sockets
BUG/MAJOR: lua: Dead lock with sockets
BUG/MEDIUM: lua/socket: Notification error
BUG/MEDIUM: lua/socket: Sheduling error on write: may dead-lock
BUG/MEDIUM: lua/socket: Buffer error, may segfault
MINOR: lua: Increase debug information
BUG/MAJOR: ssl: Random crash with cipherlist capture
BUG/MAJOR: ssl: OpenSSL context is stored in non-reserved memory slot
William Lallemand (5):
BUG/MEDIUM: cache: don't cache when an Authorization header is present
BUG/MEDIUM: threads: handle signal queue only in thread 0
BUG/MINOR: don't ignore SIG{BUS,FPE,ILL,SEGV} during signal processing
BUG/MINOR: signals: ha_sigmask macro for multithreading
BUG/MAJOR: map: fix a segfault when using http-request set-map
Willy Tarreau (3):
BUG/MINOR: ssl/lua: prevent lua from affecting automatic maxconn
computation
BUG/BUILD: threads: unbreak build without threads
BUG/BUILD: fd: fix typo causing a warning when threads are disabled
sada (1):
BUG/MINOR: lua: Socket.send threw runtime error: 'close' needs 1
arguments.
---
--
William Lallemand
Re: Observations about reloads and DNS SRV records
Hi Tait, There is a first thing you can do to prevent this from happening, but this might be complicated to implement. If you force a backend id in the configuration file, the problem won't exist anymore... backend be server-template srv 5 _http._tcp.srv.tld resolvers dnsserver resolve-prefer ipv4 check id 5 backend be2 server-template srv 5 _http._tcp.srv.tld resolvers dnsserver resolve-prefer ipv4 check id 4 When I change the order of those 2 backends, my config reloads properly. Note that I'm currently investigating a way to make this happen without enforcing the backend id. Baptiste
Re: HAProxy 1.8 built with rpath'd openssl links ok; but `haproxy -vv` reports "Built with" and "Running on" conflict
Hi On 6/21/18 10:50 PM, Willy Tarreau wrote:> Well, first please ensure you're building latest fixes and not and old > version otherwise you'll expose yourself to all these known and now fixed > bugs : oops, missed that! :-/ switching to 'latest' in branch, v1.8.9, still see the same in "-vv" output, ldd `which haproxy` | egrep "ssl|crypto" libssl.so.1.1 => /usr/local/openssl11/lib64/libssl.so.1.1 (0x7f6a21894000) libcrypto.so.1.1 => /usr/local/openssl11/lib64/libcrypto.so.1.1 (0x7f6a213f9000) haproxy -vv HA-Proxy version 1.8.9-83616ec 2018/05/18 Copyright 2000-2018 Willy Tarreau Build options : TARGET = linux2628 CPU = generic CC = gcc CFLAGS = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement -fwrapv -fno-strict-overflow -Wno-format-truncation -Wno-null-dereference -Wno-unused-label OPTIONS = USE_OPENSSL=1 USE_SYSTEMD=1 USE_PCRE2=1 USE_PCRE2_JIT=1 Default settings : maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200 Built with OpenSSL version : OpenSSL 1.1.0h-fips 27 Mar 2018 Running on OpenSSL version : OpenSSL 1.1.0h 27 Mar 2018 ... Again, not sure if it's a 'real' problem, or just a report artifact.
PATCHES: new resolve-accept-dup-ip server flag
Hi,
Purpose of this flag is to allow IP address duplication in a backend (on a
per server basis) when the runtime resolver is in operation.
By default, the runtime resolver prevent an IP address being affected to 2
servers when they are resolved by the same hostname.
>From the 4 patches in attachment, 3 of them can be backported. They are
more cosmetic than anything, but it took me some time to figure out who,
from the code or the comment was wrong...
This feature was requested by Ryuzaki on discource. I just provide him the
patches, so waiting for his feedback.
Baptiste
From 0490a231d457236db92b73e98df57b4f6bad2b01 Mon Sep 17 00:00:00 2001
From: Baptiste Assmann
Date: Fri, 22 Jun 2018 15:04:43 +0200
Subject: [PATCH 4/4] MINOR: dns: new flag to allow IP address duplication
By default, HAProxy's DNS resolution at runtime ensure that there is no
IP address duplication in a backend (for servers being resolved by the
same hostname).
There are a few cases where people want, on purpose, to disable this
feature.
This patch introduces a new server side flag for this purpose:
"resolve-accept-dup-ip".
---
doc/configuration.txt | 21 +
include/types/dns.h | 2 ++
src/dns.c | 6 +-
src/server.c | 6 ++
4 files changed, 34 insertions(+), 1 deletion(-)
diff --git a/doc/configuration.txt b/doc/configuration.txt
index e901d7e..85e1c9d 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -11682,6 +11682,27 @@ rise
after consecutive successful health checks. This value defaults to 2
if unspecified. See also the "check", "inter" and "fall" parameters.
+resolve-accept-dup-ip
+ By default, HAProxy prevents IP address duplication in a backend when DNS
+ resolution at runtime is in operation.
+ That said, for some cases, it makes sense that two servers (in the same
+ backend, being resolved by the same FQDN) have the same IP address.
+ For such case, simply enable this flag.
+
+ Example:
+backend b_myapp
+ default-server init-addr none resolvers dns
+ server s1 myapp.example.com:80 check resolve-accept-dup-ip
+ server s2 myapp.example.com:81 check resolve-accept-dup-ip
+
+ With the flag resolve-accept-dup-ip set:
+ * if the nameserver returns a single IP address, then both servers will use
+it
+ * If the nameserver returns 2 IP addresses, then each server will pick up a
+different address
+
+ Default value: not set
+
resolve-prefer
When DNS resolution is enabled for a server and multiple IP addresses from
different families are returned, HAProxy will prefer using an IP address
diff --git a/include/types/dns.h b/include/types/dns.h
index 9b1d08d..488d399 100644
--- a/include/types/dns.h
+++ b/include/types/dns.h
@@ -245,6 +245,8 @@ struct dns_options {
} mask;
} pref_net[SRV_MAX_PREF_NET];
int pref_net_nb; /* The number of registered prefered networks. */
+ int accept_duplicate_ip; /* flag to indicate whether the associated object can use an IP address
+already set to an other object of the same group */
};
/* Resolution structure associated to single server and used to manage name
diff --git a/src/dns.c b/src/dns.c
index 018c86a..77bf5c0 100644
--- a/src/dns.c
+++ b/src/dns.c
@@ -962,8 +962,10 @@ int dns_get_ip_from_response(struct dns_response_packet *dns_p,
int currentip_sel;
int j;
int score, max_score;
+ int allowed_duplicated_ip;
family_priority = dns_opts->family_prio;
+ allowed_duplicated_ip = dns_opts->accept_duplicate_ip;
*newip = newip4 = newip6 = NULL;
currentip_found = 0;
*newip_sin_family = AF_UNSPEC;
@@ -1027,7 +1029,9 @@ int dns_get_ip_from_response(struct dns_response_packet *dns_p,
* member of a group. If not, the score should be incremented
* by 2. */
if (owner && snr_check_ip_callback(owner, ip, _type)) {
- continue;
+ if (!allowed_duplicated_ip) {
+continue;
+ }
} else {
score += 2;
}
diff --git a/src/server.c b/src/server.c
index 277d140..b3bd9e2 100644
--- a/src/server.c
+++ b/src/server.c
@@ -1544,6 +1544,7 @@ static void srv_settings_cpy(struct server *srv, struct server *src, int srv_tmp
if (src->resolvers_id != NULL)
srv->resolvers_id = strdup(src->resolvers_id);
srv->dns_opts.family_prio = src->dns_opts.family_prio;
+ srv->dns_opts.accept_duplicate_ip = src->dns_opts.accept_duplicate_ip;
if (srv->dns_opts.family_prio == AF_UNSPEC)
srv->dns_opts.family_prio = AF_INET6;
memcpy(srv->dns_opts.pref_net,
@@ -2082,6 +2083,7 @@ int parse_server(const char *file, int linenum, char **args, struct proxy *curpr
newsrv = >defsrv;
cur_arg = 1;
newsrv->dns_opts.family_prio = AF_INET6;
+ newsrv->dns_opts.accept_duplicate_ip = 0;
}
while (*args[cur_arg]) {
@@ -2177,6 +2179,10 @@ int parse_server(const char *file, int linenum, char **args, struct proxy *curpr
newsrv->resolvers_id = strdup(args[cur_arg + 1]);
cur_arg += 2;
}
+ else if (!strcmp(args[cur_arg],
Re: remaining process after (seamless) reload
On Fri, Jun 22, 2018 at 2:28 PM William Lallemand wrote: > The seamless reload only transfers the listeners, the remaining connections > are > handled in the leaving process. It would be complicated to handle them in a > new > process with a different configuration. Thanks for the clarification. I am pushing this version on some more machines. -- William
Re: remaining process after (seamless) reload
On Fri, Jun 22, 2018 at 02:41:44PM +0200, Christopher Faulet wrote: > You're right, removing the thread from all_threads_mask when it exits is a > good idea. Here a patch to do so. merged as well, thanks! Willy
Re: remaining process after (seamless) reload
Le 20/06/2018 à 18:29, Willy Tarreau a écrit : On Wed, Jun 20, 2018 at 04:42:58PM +0200, Christopher Faulet wrote: When HAProxy is shutting down, it exits the polling loop when there is no jobs anymore (jobs == 0). When there is no thread, it works pretty well, but when HAProxy is started with several threads, a thread can decide to exit because jobs variable reached 0 while another one is processing a task (e.g. a health-check). At this stage, the running thread could decide to request a synchronization. But because at least one of them has already gone, the others will wait infinitly in the sync point and the process will never die. Just a comment on this last sentence, I think this is the root cause of the problem : a thread quits and its thread_mask bit doesn't disappear. In my opinion if we're looping, it's precisely because there's no way by looking at the all_threads_mask if some threads are missing. Thus I think that a more reliable long term solution would require that each exiting thread does at least "all_threads_mask &= ~tid_bit". Now I have no idea whether or not the current sync point code is compatible with this nor if this will be sufficient, but I'm pretty sure that over time we'll have to go this way to fix this inconsistency. Hi Willy, You're right, removing the thread from all_threads_mask when it exits is a good idea. Here a patch to do so. Thanks, -- Christopher Faulet >From cf238c63d4db1567756d388c7473122451adef17 Mon Sep 17 00:00:00 2001 From: Christopher Faulet Date: Thu, 21 Jun 2018 09:57:39 +0200 Subject: [PATCH] MINOR: threads: Be sure to remove threads from all_threads_mask on exit When HAProxy is started with several threads, Each running thread holds a bit in the bitfiled all_threads_mask. This bitfield is used here and there to check which threads are registered to take part in a specific processing. So when a thread exits, it seems normal to remove it from all_threads_mask. --- src/haproxy.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/haproxy.c b/src/haproxy.c index 5906f7989..11d1d47ce 100644 --- a/src/haproxy.c +++ b/src/haproxy.c @@ -2473,6 +2473,8 @@ static void *run_thread_poll_loop(void *data) list_for_each_entry(ptdf, _thread_deinit_list, list) ptdf->fct(); + HA_ATOMIC_AND(_threads_mask, ~tid_bit); + #ifdef USE_THREAD if (tid > 0) pthread_exit(NULL); -- 2.17.1
Re: remaining process after (seamless) reload
On Fri, Jun 22, 2018 at 12:03:22PM +0200, William Dauchy wrote: > On Thu, Jun 21, 2018 at 5:21 PM William Lallemand > wrote: > > Once you are sure this is not a bug and that a client is still connected, > > you > > could use the keyword 'hard-stop-after' to force a hard stop. > > After double checking some cases, indeed there are still a few > remaining established connections. I however was expecting them to be > transmitted on the new process through the seamless reload, but I am > probably mixing things. > Apart from that, no other issues seen so far. > The seamless reload only transfers the listeners, the remaining connections are handled in the leaving process. It would be complicated to handle them in a new process with a different configuration. Thanks for the feedback! -- William Lallemand
Re: remaining process after (seamless) reload
On Thu, Jun 21, 2018 at 5:21 PM William Lallemand wrote: > Once you are sure this is not a bug and that a client is still connected, you > could use the keyword 'hard-stop-after' to force a hard stop. After double checking some cases, indeed there are still a few remaining established connections. I however was expecting them to be transmitted on the new process through the seamless reload, but I am probably mixing things. Apart from that, no other issues seen so far. -- William
Re: maxsslconn vs maxsslrate
On Thu, Jun 07, 2018 at 02:47:01PM +0530, Mihir Shirali wrote: > Hi Alexander, > > I have looked at the link. What I am looking for is an answer to the > difference between maxsslconn and maxsslrate. The former does not result in > CPU savings while the latter does. That's expected : - the former limits the number of concurrent connections : it will refuse to allocate more SSL sessions than configured. The problem is that this can only be done after the TCP connection is accepted, so extraneous connections will be rejected instead of being delayed. This is used to limit the amount of memory used by SSL. - the latter limits the rate at which incoming connections are accepted on all SSL listeners. Once the limit is reached, new connections will not be accepted anymore until the average accept rate over the last sliding second falls lower than this value. This effectively results in a smoothing of the SSL traffic. For example if you accept only 200 conns/s you'll ultimately see one connection accepted every 5 ms. In both cases there are limitations though : - with maxsslconn, some users will not be able to connect at all if the limit is reached ; - with maxsslrate, some users will experience a slowdown so that the max rate you have configured is respected. Overall you should use this with high enough values to protect your system but not really for QoS. Regards, willy
Re: remaining process after (seamless) reload
On Thu, Jun 21, 2018 at 05:20:19PM +0200, William Lallemand wrote: > I'm waiting a few days for your feedback and we will probably release a new > 1.8 > version including those fixes. I've merged it in master now. Willy
Re: tcp-check expect with exclamation mark
2018-06-22 1:13 GMT+10:00 Baptiste : > > > On Thu, Jun 21, 2018 at 8:53 AM, Igor Cicimov com> wrote: > >> Hi Dmitriy, >> >> On Thu, Jun 21, 2018 at 12:45 PM, Dmitriy Kuzmin >> wrote: >> >>> Greetings >>> >>> I’m using haproxy to load balance readonly queries between redis slaves. >>> I want to use health check system to exclude slaves from load balancing, >>> that are in a process of sync with master. >>> The idea is to look for a string “master_sync_in_progress:1” in response >>> to “info replication”. >>> >> >> Is this _exactly_ what is returned? >> >> >>> If this string is found then backend should be marked as down. >>> >>> I’m trying to use “tcp-check expect ! string” (with exclamation mark >>> [!]) to get this working, but backends are permanently down regardless of >>> sync status. >>> During sync (slave’s response contains “master_sync_in_progress:1”) >>> health check status is “L7RSP,TCPCHK matched unwanted content >>> 'master_sync_in_progress:1' at step 5”. >>> However, when slave’s response contains “master_sync_in_progress:0” >>> (sync finished) health check status is “L7TOUT, at step 5 of tcp-check >>> (expect string 'master_sync_in_progress:1’)”. >>> >>> Does negation with exclamation mark (!) work with tcp-check at all? >>> >>> I’ve observed this behaviour in haproxy 1.5.18, 1.7.11 and 1.8.9 >>> >>> >>> Example configuration: >>> >>> global >>> log 127.0.0.1 local2 >>> chroot/var/lib/haproxy >>> pidfile /var/run/haproxy.pid >>> maxconn 4000 >>> user haproxy >>> group haproxy >>> daemon >>> >>> defaults >>> modetcp >>> log global >>> option tcplog >>> option dontlognull >>> option redispatch >>> retries 3 >>> timeout http-request10s >>> timeout queue 1m >>> timeout connect 10s >>> timeout client 1m >>> timeout server 1m >>> timeout http-keep-alive 10s >>> timeout check 10s >>> maxconn 3000 >>> >>> frontend redis_reads_ft >>> bind/var/run/haproxy/redis_reads >>> use_backend redis_reads_bk >>> >>> backend redis_reads_bk >>> option log-health-checks >>> balance roundrobin >>> >>> optiontcp-check >>> tcp-check connect >>> tcp-check send PING\r\n >>> tcp-check expect string +PONG >>> tcp-check send info\ replication\r\n >>> tcp-check expect ! string >>> >>> master_sync_in_progress:1 >>> >> >> Try using *rstring* intead of *string*. I that fails too try escaping >> the column like "master_sync_in_progress\:1" >> >> tcp-check send QUIT\r\n >>> tcp-check expect string +OK >>> >>> server sc-redis1_63811 10.10.68.61:63811 check >>> server sc-redis1_63812 10.10.68.61:63812 check >>> server sc-redis1_63813 10.10.68.61:63813 check >>> >>> >>> Best regards, >>> Dmitriy Kuzmin >>> >> >> > > I'm not sure what string you're trying to match. Could you paste the > output of "info replication" somewhere on pastebin or gist? > > Baptiste > Greetings Thank you for your reply Here is full output from "healthy" backend: https://pastebin.com/in9yH4e4 Here is full output from "unhealthy" backend: https://pastebin.com/FSPaiKW0 I'm trying to match a string "master_sync_in_progress:1" which is contained only in "unhealthy" backend response. With this rules (mention the exclamation mark): tcp-check connect tcp-check send info\ replication\r\n tcp-check expect ! string master_sync_in_progress:1 i expect that only backends, whose response contains "master_sync_in_progress:1" will be marked as DOWN. However, both healthy and unhealthy backends are marked as DOWN. Unhealthy backends correctly marked as DOWN because there is a match: "matched unwanted content 'master_sync_in_progress:1 at step 5" But healthy backends are marked as DOWN too. Because of “L7TOUT, at step 5 of tcp-check (expect string 'master_sync_in_progress:1’)”. And i'm trying to figure out why is that. Any guess? Best regards, Dmitriy Kuzmin
