Re: 1.9.6: SIGFPE in fwrr_update_position
Hi! It seems to me there is something wrong with this patch: for some reason process stops responding with 100% CPU used by all threads. Backtrace: (gdb) thread apply all bt Thread 4 (Thread 0x7fdf68c9c700 (LWP 615744)): #0 0x564fc9a61990 in fwrr_update_server_weight (srv=0x564fcb5014b0) at src/lb_fwrr.c:198 #1 0x564fc99b5363 in srv_update_status (s=0x564fcb5014b0) at src/server.c:4923 #2 0x564fc99b46e2 in server_recalc_eweight (sv=sv@entry=0x564fcb5014b0, must_update=must_update@entry=1) at src/server.c:1310 #3 0x564fc99b6ca2 in server_parse_weight_change_request (sv=sv@entry=0x564fcb5014b0, weight_str=weight_str@entry=0x564fcb50a1d0 "68%") at src/server.c:1356 #4 0x564fc99c1f3c in __event_srv_chk_r (cs=cs@entry=0x7fdf62885e20) at src/checks.c:1114 #5 0x564fc99c5000 in event_srv_chk_io (t=, ctx=0x564fcb501b70, state=) at src/checks.c:730 #6 0x564fc9a56bb2 in process_runnable_tasks () at src/task.c:390 #7 0x564fc99ccba0 in run_poll_loop () at src/haproxy.c:2652 #8 run_thread_poll_loop (data=) at src/haproxy.c:2717 #9 0x7fdf6c7326ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #10 0x7fdf6b70241d in clone () from /lib/x86_64-linux-gnu/libc.so.6 Thread 3 (Thread 0x7fdf6949d700 (LWP 615743)): #0 0x564fc9a61e7a in fwrr_get_next_server (p=0x564fcabd8e60, srvtoavoid=srvtoavoid@entry=0x0) at src/lb_fwrr.c:528 #1 0x564fc9a11fa8 in assign_server (s=s@entry=0x7fdf54860b80) at src/backend.c:673 #2 0x564fc9a12b07 in assign_server_and_queue (s=s@entry=0x7fdf54860b80) at src/backend.c:963 #3 0x564fc9a15e07 in assign_server_and_queue (s=0x7fdf54860b80) at include/proto/freq_ctr.h:55 #4 srv_redispatch_connect (s=s@entry=0x7fdf54860b80) at src/backend.c:1621 #5 0x564fc9988836 in sess_prepare_conn_req (s=0x7fdf54860b80) at src/stream.c:1163 #6 process_stream (t=, context=0x7fdf54860b80, state=) at src/stream.c:2310 #7 0x564fc9a56807 in process_runnable_tasks () at src/task.c:387 #8 0x564fc99ccba0 in run_poll_loop () at src/haproxy.c:2652 #9 run_thread_poll_loop (data=) at src/haproxy.c:2717 #10 0x7fdf6c7326ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #11 0x7fdf6b70241d in clone () from /lib/x86_64-linux-gnu/libc.so.6 Thread 2 (Thread 0x7fdf69c9e700 (LWP 615742)): #0 0x564fc9a61e7a in fwrr_get_next_server (p=0x564fcabd8e60, srvtoavoid=srvtoavoid@entry=0x0) at src/lb_fwrr.c:528 #1 0x564fc9a11fa8 in assign_server (s=s@entry=0x7fdf667a3690) at src/backend.c:673 #2 0x564fc9a12b07 in assign_server_and_queue (s=s@entry=0x7fdf667a3690) at src/backend.c:963 #3 0x564fc9a15e07 in assign_server_and_queue (s=0x7fdf667a3690) at include/proto/freq_ctr.h:55 #4 srv_redispatch_connect (s=s@entry=0x7fdf667a3690) at src/backend.c:1621 #5 0x564fc9988836 in sess_prepare_conn_req (s=0x7fdf667a3690) at src/stream.c:1163 #6 process_stream (t=, context=0x7fdf667a3690, state=) at src/stream.c:2310 #7 0x564fc9a56807 in process_runnable_tasks () at src/task.c:387 #8 0x564fc99ccba0 in run_poll_loop () at src/haproxy.c:2652 #9 run_thread_poll_loop (data=) at src/haproxy.c:2717 #10 0x7fdf6c7326ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 ---Type to continue, or q to quit--- #11 0x7fdf6b70241d in clone () from /lib/x86_64-linux-gnu/libc.so.6 Thread 1 (Thread 0x7fdf6cf27180 (LWP 615741)): #0 fwrr_get_server_from_group (grp=0x564fcabd9b88) at src/lb_fwrr.c:464 #1 fwrr_get_next_server (p=0x564fcabd8e60, srvtoavoid=srvtoavoid@entry=0x0) at src/lb_fwrr.c:556 #2 0x564fc9a11fa8 in assign_server (s=s@entry=0x564fd48c4f90) at src/backend.c:673 #3 0x564fc9a12b07 in assign_server_and_queue (s=s@entry=0x564fd48c4f90) at src/backend.c:963 #4 0x564fc9a15e07 in assign_server_and_queue (s=0x564fd48c4f90) at include/proto/freq_ctr.h:55 #5 srv_redispatch_connect (s=s@entry=0x564fd48c4f90) at src/backend.c:1621 #6 0x564fc9988836 in sess_prepare_conn_req (s=0x564fd48c4f90) at src/stream.c:1163 #7 process_stream (t=, context=0x564fd48c4f90, state=) at src/stream.c:2310 #8 0x564fc9a56807 in process_runnable_tasks () at src/task.c:387 #9 0x564fc99ccba0 in run_poll_loop () at src/haproxy.c:2652 #10 run_thread_poll_loop (data=) at src/haproxy.c:2717 #11 0x564fc992779c in main (argc=, argv=) at src/haproxy.c:3379 ср, 17 апр. 2019 г. в 05:11, Willy Tarreau : > Hi Maksim, > > On Tue, Apr 16, 2019 at 07:28:28AM +0200, Willy Tarreau wrote: > > > So I agree upon another thread activity. The unique thing about > > > these servers - all of them use haproxy-agent to set up weights of > their > > > backends. Other instances with no haproxy-agent in their configs don't > > > produce cores. > > > > Great, this will definitely help me validate my hypothesis. I'm not sure > > the fix will be easy but I'm back to this. > > OK so I could finally figure what the problem was and fix it. The upper > level function used to expect to be called with the
Re: [PATCH v2] MINOR: travis: Extend .travis.yml
On Wed, Apr 24, 2019 at 12:30:36AM +0500, ??? wrote: > > http://git.haproxy.org/?p=haproxy.git;a=commit;h=333939c2eee8f9dbfe11f66073e52892b38347e7 > > > > This one was just found and fixed by Fred. Just pushed as commit bed883ab. > > > > there're seem to be more regressions. I can run newest reg-tests against > parent of that commit, but they fail on bed883ab OK. Thank you Ilya. At this point I have no idea since I don't see these ones. Willy
Re: [PATCH v2] MINOR: travis: Extend .travis.yml
вт, 23 апр. 2019 г. в 23:25, Willy Tarreau : > On Tue, Apr 23, 2019 at 10:31:05PM +0500, ??? wrote: > > Hello, Baptise! > > > > we have enabled travis-ci builds on haproxy master branch. seems we have > a > > regression here: > > > > > http://git.haproxy.org/?p=haproxy.git;a=commit;h=333939c2eee8f9dbfe11f66073e52892b38347e7 > > This one was just found and fixed by Fred. Just pushed as commit bed883ab. > there're seem to be more regressions. I can run newest reg-tests against parent of that commit, but they fail on bed883ab > > Cheers, > Willy >
Re: [PATCH v2] MINOR: travis: Extend .travis.yml
On Tue, Apr 23, 2019 at 10:31:05PM +0500, ??? wrote: > Hello, Baptise! > > we have enabled travis-ci builds on haproxy master branch. seems we have a > regression here: > > http://git.haproxy.org/?p=haproxy.git;a=commit;h=333939c2eee8f9dbfe11f66073e52892b38347e7 This one was just found and fixed by Fred. Just pushed as commit bed883ab. Cheers, Willy
Re: [PATCH v2] MINOR: travis: Extend .travis.yml
yep, I'm going to invest some time in building openssl matrix on travis-ci, i.e. openssl-1.0, 1.0.2, 1.1.1, libressl, ... вт, 23 апр. 2019 г. в 22:31, Willy Tarreau : > On Tue, Apr 23, 2019 at 10:11:09PM +0500, ??? wrote: > > I want to bisect in order to find offending commit. > > we've got regression. it need to be fixed, no need to mark it as "allowed > > failure" > > Got it. We've spotted issues with SSL that affect openssl < 1.1, and > very likely related to the recent build fixes after the changes in BIO. > I'll wait for Olivier to have a look at this. So if you're bisecting > this, maybe wait a day or two and don't waste your time. > > Cheers, > Willy >
Re: [PATCH v2] MINOR: travis: Extend .travis.yml
this particular regression is reproduced on my fedora laptop, which is openssl-1.1.X вт, 23 апр. 2019 г. в 22:31, Willy Tarreau : > On Tue, Apr 23, 2019 at 10:11:09PM +0500, ??? wrote: > > I want to bisect in order to find offending commit. > > we've got regression. it need to be fixed, no need to mark it as "allowed > > failure" > > Got it. We've spotted issues with SSL that affect openssl < 1.1, and > very likely related to the recent build fixes after the changes in BIO. > I'll wait for Olivier to have a look at this. So if you're bisecting > this, maybe wait a day or two and don't waste your time. > > Cheers, > Willy >
Re: [PATCH v2] MINOR: travis: Extend .travis.yml
Hello, Baptise! we have enabled travis-ci builds on haproxy master branch. seems we have a regression here: http://git.haproxy.org/?p=haproxy.git;a=commit;h=333939c2eee8f9dbfe11f66073e52892b38347e7 can you have a look please ? failing build (we have enabled them just recently): https://travis-ci.com/haproxy/haproxy/builds/109289462 вт, 23 апр. 2019 г. в 22:11, Илья Шипицин : > I want to bisect in order to find offending commit. > we've got regression. it need to be fixed, no need to mark it as "allowed > failure" > > вт, 23 апр. 2019 г. в 22:00, Willy Tarreau : > >> On Tue, Apr 23, 2019 at 09:20:52PM +0500, ??? wrote: >> > let us wait for a while and do not enable "allow failures". >> >> That's unclear to me Ilya, do you mean you prefer that we wait a bit >> more without taking it, that's it ? I'm fine with all options. >> >> Willy >> >
Re: [PATCH v2] MINOR: travis: Extend .travis.yml
On Tue, Apr 23, 2019 at 10:11:09PM +0500, ??? wrote: > I want to bisect in order to find offending commit. > we've got regression. it need to be fixed, no need to mark it as "allowed > failure" Got it. We've spotted issues with SSL that affect openssl < 1.1, and very likely related to the recent build fixes after the changes in BIO. I'll wait for Olivier to have a look at this. So if you're bisecting this, maybe wait a day or two and don't waste your time. Cheers, Willy
Re: [PATCH v2] MINOR: travis: Extend .travis.yml
I want to bisect in order to find offending commit. we've got regression. it need to be fixed, no need to mark it as "allowed failure" вт, 23 апр. 2019 г. в 22:00, Willy Tarreau : > On Tue, Apr 23, 2019 at 09:20:52PM +0500, ??? wrote: > > let us wait for a while and do not enable "allow failures". > > That's unclear to me Ilya, do you mean you prefer that we wait a bit > more without taking it, that's it ? I'm fine with all options. > > Willy >
Re: [PATCH v2] MINOR: travis: Extend .travis.yml
On Tue, Apr 23, 2019 at 09:20:52PM +0500, ??? wrote: > let us wait for a while and do not enable "allow failures". That's unclear to me Ilya, do you mean you prefer that we wait a bit more without taking it, that's it ? I'm fine with all options. Willy
leak of handle to /dev/urandom since 1.8?
Hi, We've noticed that our haproxy processes accumulate many file descriptors to /dev/urandom over time. This coincidences with reloads; rnewson@lb2:~$ sudo service haproxy restart rnewson@lb2:~$ sudo ls -l /proc/$(pgrep -o haproxy)/fd | grep -c '/dev/urandom' 1 rnewson@lb2:~$ sudo ls -l /proc/$(pgrep -o haproxy)/fd | grep -c '/dev/urandom' 1 rnewson@lb2:~$ sudo service haproxy reload rnewson@lb2:~$ sudo ls -l /proc/$(pgrep -o haproxy)/fd | grep -c '/dev/urandom' 2 rnewson@lb2:~$ sudo ls -l /proc/$(pgrep -o haproxy)/fd | grep -c '/dev/urandom' 2 rnewson@lb2:~$ sudo service haproxy reload rnewson@lb2:~$ sudo ls -l /proc/$(pgrep -o haproxy)/fd | grep -c '/dev/urandom' 3 rnewson@lb2:~$ sudo service haproxy reload rnewson@lb2:~$ sudo ls -l /proc/$(pgrep -o haproxy)/fd | grep -c '/dev/urandom' 4 We've seen a node with over 20,000 open file descriptors to this same file. We are running under systemd. Our config is quite complicated so I'm not posting it here just yet, I'd appreciate guidance on which parts might be relevant. We do use "expose-fd listeners" to hand over open sockets cleanly during reload but nothing else seems obviously related. We see this with haproxy 1.8.16 and 1.9.6. We have a few deployments of 1.7.9 where the count remains at 0 (which is also odd given we use TLS exclusively). Any suggestions? B. -- Robert Samuel Newson rnew...@apache.org
Re: [PATCH v2] MINOR: travis: Extend .travis.yml
let us wait for a while and do not enable "allow failures". вт, 23 апр. 2019 г. в 20:24, Willy Tarreau : > On Tue, Apr 23, 2019 at 04:41:44PM +0200, Tim Düsterhus wrote: > > List, > > > > Am 23.04.19 um 16:40 schrieb Tim Duesterhus: > > > [...] > > > > Sorry for the noise. This v2 only differs by a non-unicode author name. > > The 'ü' slipped in, because I initially developed this patch using > GitHub. > > OK :-) > > I'm tempted to merge it right now just to see how it works. Let's wait a > few hours first to try to get some news from Ilya, otherwise I'll take it. > In the worst case Ilya can simply suggest to fix it tomorrow. > > Thanks, > Willy >
Re: [PATCH v2] MINOR: travis: Extend .travis.yml
On Tue, Apr 23, 2019 at 04:41:44PM +0200, Tim Düsterhus wrote: > List, > > Am 23.04.19 um 16:40 schrieb Tim Duesterhus: > > [...] > > Sorry for the noise. This v2 only differs by a non-unicode author name. > The 'ü' slipped in, because I initially developed this patch using GitHub. OK :-) I'm tempted to merge it right now just to see how it works. Let's wait a few hours first to try to get some news from Ilya, otherwise I'll take it. In the worst case Ilya can simply suggest to fix it tomorrow. Thanks, Willy
Re: [PATCH v2] MINOR: travis: Extend .travis.yml
List, Am 23.04.19 um 16:40 schrieb Tim Duesterhus: > [...] Sorry for the noise. This v2 only differs by a non-unicode author name. The 'ü' slipped in, because I initially developed this patch using GitHub. Best regards Tim Düsterhus
[PATCH v2] MINOR: travis: Extend .travis.yml
This commit extends the Travis CI configuration to build HAProxy with
gcc on Linux, clang on Mac and cleans up the build flag configuration to
be easier extendable.
Note: At the moment HAProxy fails on Travis for two configurations
- on Linux with build flags enabled
- on OS X
Thus these two jobs are set to 'allow_failures' for now. Once the issues
have been resolved this special configuration should be removed.
---
.travis.yml | 47 ---
1 file changed, 36 insertions(+), 11 deletions(-)
diff --git a/.travis.yml b/.travis.yml
index c8937f377..0e26fe7e2 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,24 +1,49 @@
-sudo: required
dist: xenial
language: c
+addons:
+ apt:
+packages:
+- liblua5.3-dev
+
matrix:
include:
-# - os: linux
-#compiler: gcc
-#env: TARGET=linux2628
+ - os: linux
+compiler: gcc
+env: TARGET=linux2628 FLAGS=
+ - os: linux
+compiler: gcc
+env: TARGET=linux2628 FLAGS="USE_ZLIB=1 USE_PCRE=1 USE_LUA=1 USE_OPENSSL=1"
- os: linux
compiler: clang
-env: TARGET=linux2628 USE_THREAD=1 USE_OPENSSL=1 USE_PCRE=1 USE_ZLIB=1
USE_GETADDRINFO=1
-# - os: osx
-#compiler: clang
-#env: TARGET=osx SSL_LIB=/usr/local/opt/openssl/lib
SSL_INC=/usr/local/opt/openssl/include TMPDIR=/var/tmp
+env: TARGET=linux2628 FLAGS=
+ - os: osx
+compiler: clang
+env: TARGET=generic FLAGS=
+ allow_failures:
+- os: osx
+- env: TARGET=linux2628 FLAGS="USE_ZLIB=1 USE_PCRE=1 USE_LUA=1
USE_OPENSSL=1"
install:
- git clone https://github.com/VTest/VTest.git ../vtest
- - make -C ../vtest
+ # Special flags due to: https://github.com/vtest/VTest/issues/12
+ - make -C ../vtest FLAGS="-O2 -s -Wall"
+
+before_script:
+ # This is a fix for the super long TMPDIR on Mac making
+ # the unix socket path names exceed the maximum allowed
+ # length.
+ - sed -i'.original'
'/TESTDIR=.*haregtests/s/haregtests-.*XX/regtest.XXX/'
scripts/run-regtests.sh
script:
- - make CC=$CC V=1 TARGET=$TARGET USE_THREAD=${USE_THREAD}
USE_OPENSSL=${USE_OPENSSL} USE_PCRE=${USE_PCRE} USE_ZLIB=${USE_ZLIB}
USE_GETADDRINFO=${USE_GETADDRINFO}
- - make reg-tests PATH=${PATH}:${PWD}/../vtest VTEST_PROGRAM="../vtest/vtest
-v"
+ - make CC=$CC V=1 TARGET=$TARGET $FLAGS
+ - ./haproxy -vv
+ - env VTEST_PROGRAM=../vtest/vtest make reg-tests
+
+after_failure:
+ - |
+for folder in ${TMPDIR:-/tmp}/*regtest*/vtc.*; do
+ cat $folder/INFO
+ cat $folder/LOG
+done
--
2.21.0
[PATCH] MINOR: travis: Extend .travis.yml
From: Tim Düsterhus
This commit extends the Travis CI configuration to build HAProxy with
gcc on Linux, clang on Mac and cleans up the build flag configuration to
be easier extendable.
Note: At the moment HAProxy fails on Travis for two configurations
- on Linux with build flags enabled
- on OS X
Thus these two jobs are set to 'allow_failures' for now. Once the issues
have been resolved this special configuration should be removed.
---
.travis.yml | 47 ---
1 file changed, 36 insertions(+), 11 deletions(-)
diff --git a/.travis.yml b/.travis.yml
index c8937f377..0e26fe7e2 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,24 +1,49 @@
-sudo: required
dist: xenial
language: c
+addons:
+ apt:
+packages:
+- liblua5.3-dev
+
matrix:
include:
-# - os: linux
-#compiler: gcc
-#env: TARGET=linux2628
+ - os: linux
+compiler: gcc
+env: TARGET=linux2628 FLAGS=
+ - os: linux
+compiler: gcc
+env: TARGET=linux2628 FLAGS="USE_ZLIB=1 USE_PCRE=1 USE_LUA=1 USE_OPENSSL=1"
- os: linux
compiler: clang
-env: TARGET=linux2628 USE_THREAD=1 USE_OPENSSL=1 USE_PCRE=1 USE_ZLIB=1
USE_GETADDRINFO=1
-# - os: osx
-#compiler: clang
-#env: TARGET=osx SSL_LIB=/usr/local/opt/openssl/lib
SSL_INC=/usr/local/opt/openssl/include TMPDIR=/var/tmp
+env: TARGET=linux2628 FLAGS=
+ - os: osx
+compiler: clang
+env: TARGET=generic FLAGS=
+ allow_failures:
+- os: osx
+- env: TARGET=linux2628 FLAGS="USE_ZLIB=1 USE_PCRE=1 USE_LUA=1
USE_OPENSSL=1"
install:
- git clone https://github.com/VTest/VTest.git ../vtest
- - make -C ../vtest
+ # Special flags due to: https://github.com/vtest/VTest/issues/12
+ - make -C ../vtest FLAGS="-O2 -s -Wall"
+
+before_script:
+ # This is a fix for the super long TMPDIR on Mac making
+ # the unix socket path names exceed the maximum allowed
+ # length.
+ - sed -i'.original'
'/TESTDIR=.*haregtests/s/haregtests-.*XX/regtest.XXX/'
scripts/run-regtests.sh
script:
- - make CC=$CC V=1 TARGET=$TARGET USE_THREAD=${USE_THREAD}
USE_OPENSSL=${USE_OPENSSL} USE_PCRE=${USE_PCRE} USE_ZLIB=${USE_ZLIB}
USE_GETADDRINFO=${USE_GETADDRINFO}
- - make reg-tests PATH=${PATH}:${PWD}/../vtest VTEST_PROGRAM="../vtest/vtest
-v"
+ - make CC=$CC V=1 TARGET=$TARGET $FLAGS
+ - ./haproxy -vv
+ - env VTEST_PROGRAM=../vtest/vtest make reg-tests
+
+after_failure:
+ - |
+for folder in ${TMPDIR:-/tmp}/*regtest*/vtc.*; do
+ cat $folder/INFO
+ cat $folder/LOG
+done
--
2.21.0
H/2 via Unix Sockets fails
Hey,
we have an older setup using nbproc >1 and having a listener for the
initial tcp connection and one for the actual SSL/TLS, also using tcp
mode which then goes to the actual frontend using http mode. Each being
bound to different processes.
So here's the test config I've used:
listen h2test_tcp
mode tcp
bind :444
option tcplog
log global
server socket-444-h2test unix@/run/haproxy-444-h2test.sock
send-proxy-v2
listen h2test_tcp.tls
mode tcp
option tcplog
log global
bind unix@/run/haproxy-444-h2test.sock accept-proxy user haproxy
group haproxy mode 600 ssl crt /etc/haproxy/ssl/h2test.pem alpn
h2,http/1.1
server socket-444_2 unix@/run/haproxy-444_2-h2test.sock
send-proxy-v2
frontend some_frontend
mode http
log global
bind unix@/run/haproxy-444_2-h2test.sock id 444 accept-proxy user
haproxy group haproxy mode 600
bind :80
...
So what I'm doing is:
curl -k4vs https://127.0.0.1:444/~idl0r/ --http1.1
curl -k4vs https://127.0.0.1:444/~idl0r/ --http2
So with HTTP/1.1 I get:
public_http backend_qasl_de/qasl1 0/0/0/0/0 200 510 - - 3/1/0/0/0
0/0 {127.0.0.1:444|curl/7.64.1|} "GET / HTTP/1.1"
h2test_tcp.tls~ h2test_tcp.tls/socket-444_2 5/1/6 605 -- 2/1/0/0/0 0/0
h2test_tcp h2test_tcp/socket-444-h2test 1/0/6 3335 CD 1/1/0/0/0 0/0
With H/2:
public_http public_http/ -1/-1/-1/-1/0 400 187 - - PR-- 3/1/0/0/0
0/0 {||} ""
h2test_tcp.tls~ h2test_tcp.tls/socket-444_2 6/0/5 187 SD 2/1/0/0/0 0/0
h2test_tcp h2test_tcp/socket-444-h2test 1/0/5 2911 SD 1/1/0/0/0 0/0
curl says:
# curl -k4vs https://127.0.0.1:444/ --http2
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 444 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection:
ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-ECDSA-AES256-GCM-SHA384
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=...
* start date: Mar 1 18:00:17 2019 GMT
* expire date: May 30 18:00:17 2019 GMT
* issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after
upgrade: len=0
* Using Stream ID: 1 (easy handle 0x56087e29b770)
GET / HTTP/2
Host: 127.0.0.1:444
User-Agent: curl/7.64.1
Accept: */*
* http2 error: Remote peer returned unexpected data while we expected
SETTINGS frame. Perhaps, peer does not support HTTP/2 properly.
* Connection #0 to host 127.0.0.1 left intact
* Closing connection 0
Can anybody else confirm that? Tested with HAProxy 1.9.6.
Any ideas what might be the reason? Right now, I'd guess that's a
Problem with H/2 and those sockets on the HAProxy side.
--
Regards,
Christian Ruppert
Re: [PATCH] BUG/MAJOR: spoe: spoe_context shouldn't queue again if fragment send
Le 20/04/2019 à 12:15, Kevin Zhu a écrit : Hi I think there forgot check if the spoe_context already has fragment msg send before spoe_queue_context, it will segment fault in spoe_release_appctx. This bug is already fixed, in the upstream (3e86cec05) and HAProxy 1.9 (c3468fe1d). However, it has not been backported yet in HAProxy 1.8. Thanks, -- Christopher Faulet
Re: [PATCH] BUG/MAJOR: spoe: Rollback frequency counter to sending_rate
Le 13/04/2019 à 09:53, Kevin Zhu a écrit : The processing is really difficult to be smaller than processing_per_sec, and most msg will create a new applet, but there are already have a lot idle applets waiting for work. and the result is that the most applets is not reused. This patch should be backported to 1.9. Hi Kevin, The frequency counter was added to be more accurate and avoid creating new applets to often. So, bugs are always possible of course. but first, I'd prefer to understand what you experienced before reverting this part. I assume this patch is an attempt to fix your FDs issue on HAProxy 1.9. -- Christopher Faulet
Re: Fwd: haproxy1.9, SPOA: too many open files
Le 10/04/2019 à 08:44, Kevin Zhu a écrit : -- Forwarded message - From: *Kevin Zhu* mailto:ip0...@gmail.com>> Date: Wed, 10 Apr 2019 at 14:25 Subject: Re: haproxy1.9, SPOA: too many open files To: Christopher Faulet mailto:cfau...@haproxy.com>> Thinks reply. OS: CentOS Linux release 7.4 HW: platform: KVM; CPU: Intel Xeon E3-12xx v2 (Ivy Bridge) * 1; mem: 2048M HAProxy Make: make TARGET=linxu2628 USE_THREAD=1 SPOA: 1:cd contrib/spoa_example ; 2: set async and pipelining and fragmentation = true in souces file spoa.c 3: make 4:./spoa Benchmark: see abtest.sh Configurations files: is attached in this mail I'm testing results(result value l is approximately): HAProxy-1.8.16: multi procs single thread: CPU: HAProxy: 10% * 8; spoa: 4% - 7% TCP HAProxy and spoa(netstat -anpt | grep 12345 | grep EST | wc -l): 150 - 400 Time taken for tests: 67 seconds multi threads single thread: CPU: HAProxy: 85% - 90%; spoa: 3% - 5% TCP HAProxy and spoa(netstat -anpt | grep 12345 | grep EST | wc -l): 100 - 200 Time taken for tests: 53 seconds HAProxy-1.9.6: multi procs single thread: CPU: HAProxy: 9% * 8; spoa: 24% - 27% TCP HAProxy and spoa(netstat -anpt | grep 12345 | grep EST | wc -l):8050 Time taken for tests: 99 seconds multi threads single thread: CPU: HAProxy: 80%; spoa: 9% - 13% TCP HAProxy and spoa(netstat -anpt | grep 12345 | grep EST | wc -l): 7500 - 10200 Time taken for tests: 65 seconds Hi Kevin, Sorry for the delay, I was pretty busy on other bugs. So, first of all, I'm surprised. Looking at your test file (abtest.sh), you send 10k requests with 1000 concurrent connections in ~60 secs or more. It's not really fast. And the workload remains low. So there is probably a bottleneck somewhere. Could you enable the logs on HAProxy, both for the HTTP and the SPOE, and check if there is something strange. Maybe timeouts or errors. Or a timer with an high value. If possible, share it. I'm interested to take an eye on it. You could also start the spoa in debug mode. It is pretty verbose, but it can help to find the problem. Could you also use latest 1.8 and 1.9 snapshots ? -- Christopher Faulet
Re: [PATCH] FEATURE/MEDIUM: enable travis-ci builds
вт, 23 апр. 2019 г. в 14:50, Willy Tarreau : > Hi Tim, > > On Tue, Apr 23, 2019 at 11:28:22AM +0200, Tim Düsterhus wrote: > > No real management necessary. > (...) > > Ideally you should not notice anything: It means no one commited bad > > code. It might need some adjustments in the beginning, very similar to > > the issue tracker :-) > > OK that's the information I needed to have. So I'll see off-line with > Ilya what he needs exactly to do the setup. Just to be clear, that's > another area which I totally ignore and am not going to spend time > discovering because it will take me ages and my time is better spent > somewhere else. So I'll trust you guys to do the right thing (TM), to > openly report any trouble if that happens and/or to improve the setup > so that it perfectly fits your use cases. > > > But for the start the current one by Ilya is absolutely > > sufficient and a good improvement over the current situation. > > That was my understanding as well, which is why I've been asking for > more (short) info on this area of ignorance to me. And no, I wasn't > going to read the docs on the site ;-) > well, but someone has to enable travis-ci for a repository. just few simple steps, actually > > Thanks! > Willy >
Re: [PATCH] FEATURE/MEDIUM: enable travis-ci builds
Hi Tim, On Tue, Apr 23, 2019 at 11:28:22AM +0200, Tim Düsterhus wrote: > No real management necessary. (...) > Ideally you should not notice anything: It means no one commited bad > code. It might need some adjustments in the beginning, very similar to > the issue tracker :-) OK that's the information I needed to have. So I'll see off-line with Ilya what he needs exactly to do the setup. Just to be clear, that's another area which I totally ignore and am not going to spend time discovering because it will take me ages and my time is better spent somewhere else. So I'll trust you guys to do the right thing (TM), to openly report any trouble if that happens and/or to improve the setup so that it perfectly fits your use cases. > But for the start the current one by Ilya is absolutely > sufficient and a good improvement over the current situation. That was my understanding as well, which is why I've been asking for more (short) info on this area of ignorance to me. And no, I wasn't going to read the docs on the site ;-) Thanks! Willy
Re: [PATCH] http-request do-resolve
On Thu, Apr 18, 2019 at 10:49:29AM +0200, Baptiste wrote: > Hi all, Willy, > > Please find attached to this email the 4 patches for the http-request > do-resolve action I submitted a few months ago. > I integrated all feedback from Willy and also now support tcp-request > content do-resolve. Now merged, thanks. Willy
Re: [PATCH] FEATURE/MEDIUM: enable travis-ci builds
Willy, I'll chime in here. Am 23.04.19 um 11:16 schrieb Willy Tarreau: > On Mon, Apr 22, 2019 at 10:13:52PM +0500, ??? wrote: >> I can enable travis-ci by myself if you can temporarily grant me an access >> to github repo > > I'm not against doing this, it's just that I still have no idea what > the impacts are. In short, will any of us get any extra work once this > is enabled ? Does someone have to manage this once in a while ? Will it No real management necessary. You might need to update the Travis configuration every once in a while if you want to add additional build options or something about the reg test invocations change. The syntax is self explanatory I guess. > slow down git push ? Will someone get spammed with e-mails, and if so, The git push is not affected. Travis runs entirely asynchronous and reports the build back to GitHub and ... ... optionally via email if the build starts to fail (either because HAProxy fails to compile or because reg-tests fail). These email notifications can be updated / disabled in .travis.yml. See here for details: https://docs.travis-ci.com/user/notifications#configuring-email-notifications > whom ? All this might possibly be acceptable in exchange for better > testing, it's just that for me it's totally obscure and I have no idea > what we're engaging in by doing it. I just want to make sure we're not > going to rush a revert because it adds some unplanned burden to anyone > that nobody initially expected. The team of people having their hands > dirty on tools is small and I really want to be sure we're not going to > give them extra work (at least without their approval). > > So if you can explain in a few words what will happen once you enable > this, that would be great ;-) Ideally you should not notice anything: It means no one commited bad code. It might need some adjustments in the beginning, very similar to the issue tracker :-) Currently it builds only with clang on Linux with a bunch of options enabled. I have a version with additional gcc on Linux, clang on Mac and no build options. Unfortunately it fails the build: https://github.com/TimWolla/haproxy/commit/32b32f5fad0c45a14b7f1a300d811430f10391c2 and https://travis-ci.com/TimWolla/haproxy/builds/108870403 I'll have to investigate that and then provide the improved version with a patch. But for the start the current one by Ilya is absolutely sufficient and a good improvement over the current situation. Best regards Tim Düsterhus
Re: one health check instead of muli check when using master-worker model
Hi! On Mon, Apr 22, 2019 at 10:58:59AM +0200, Lukas Tribus wrote: > This is why thread support was introduced though. Using threads > instead of processes fixes this. Achieving the same performance with > threads will require some tuning, but at the end you'll end up with a > simple and scalable configuration. I wholeheartly second this! Years ago we started to work on a "health check server" and various ways to share server state information between processes. At some point I thought we'd do it over the peers protocol, just before remembering there's no inter-process communication over peers. In the end, I think that the multi-process model is just obsolete and am not going to encourage any effort towards better synchronization between processes. The complexity of the problem to solve is just too high for very limited benefits. Our threaded model shares little (almost only the control plane) and doesn't suffer from the scalability issues that usually come from too much sharing, and I think we're really balanced and able to reintegrate any multi-process based config using threads. There will be rough edges of course but I'd rather invest energy addressing them than working around problems of the previous century. cheers, Willy
Re: [PATCH] FEATURE/MEDIUM: enable travis-ci builds
Hi Ilya, On Mon, Apr 22, 2019 at 10:13:52PM +0500, ??? wrote: > I can enable travis-ci by myself if you can temporarily grant me an access > to github repo I'm not against doing this, it's just that I still have no idea what the impacts are. In short, will any of us get any extra work once this is enabled ? Does someone have to manage this once in a while ? Will it slow down git push ? Will someone get spammed with e-mails, and if so, whom ? All this might possibly be acceptable in exchange for better testing, it's just that for me it's totally obscure and I have no idea what we're engaging in by doing it. I just want to make sure we're not going to rush a revert because it adds some unplanned burden to anyone that nobody initially expected. The team of people having their hands dirty on tools is small and I really want to be sure we're not going to give them extra work (at least without their approval). So if you can explain in a few words what will happen once you enable this, that would be great ;-) Thanks! Willy
Re: [PATCH] wurfl device detection build fixes and dummy library
Hi Paul, On Fri, Apr 19, 2019 at 06:45:22PM +0200, Paul Stephen Borile wrote: > Hi Willy, > > fine for me, thanks for the adjustments and no problem backporting this to > 1.9. > I also confirm that the contact email address is working correctly. Fine thank you. I could finish the polishing (add USE_WURFL to the list of known and reported build options in the makefile) and I've reintegrated the code now. You probably don't see the value in having the dummy library, but for developers it's invaluable. I have now updated my build script to build with USE_WURFL=1 by default so that I'll now see if anything causes warnings or errors there if we touch structures that are used by your code. It would be really awesome if Device Atlas and 51Degrees could do the same, as the build coverage becomes much better with very little effort for everyone. David, Ben, if you read this, please have a look at contrib/wurfl to get an idea of what is sufficient to have your code always built by anyone. Patches welcome :-) Thanks, Willy
