Re: OSX builds in Travis
❦ 11 juillet 2020 00:48 +05, Илья Шипицин: > he-he, brew bundle is deprecated (does not work) > > https://apple.stackexchange.com/questions/148454/brew-bundle-reporting-error-unknown-command-bundle It's very old. It has been added back at some point. Here is upstream recommending its use: https://github.com/Homebrew/brew/issues/2491 However, as long as socat is not part of the packages already installed, just `brew install socat` should always work. -- It usually takes more than three weeks to prepare a good impromptu speech. -- Mark Twain
Re: proposing a haproxy 2.0.16 release (was [BUG] haproxy retries dispatch to wrong server)
Lukas, Am 10.07.20 um 21:04 schrieb Lukas Tribus: >> I finally pushed this fix in the 2.0. Note the same bug affected the HTTP >> proxy >> mode (using http_proxy option). In this case, the connection retries is now >> disabled (on the 2.0 only) because the destination address is definitely >> lost. >> It was the easiest way to work around the bug without backporting a bunch of >> sensitive patches from the 2.1. > > Given the importance and impact of this bug you just fixed (at least 5 > independent people already reported it on GH and ML) and the amount of > other important fixes already in the tree (including at least one > crash fix), I'm suggesting to release 2.0.16. Unless there are other > important open bugs with ongoing troubleshooting? > > Agreed. With 2.2 being released this is a good opportunity to release updates for all branches down to 1.9 at least to properly mark the latter as EoL. I'd also appreciate at least backports happening without a release for 1.6 and 1.7, allowing us to close off old, fixed, issues on GitHub. Best regards Tim Düsterhus
Re: [ANNOUNCE] haproxy-2.2.0
instead of disabling Lua support, is it possible to build against Lua-5.3 ? I recall there's Lua-5.3 on Fedora-33 пт, 10 июл. 2020 г. в 21:20, Ryan O'Hara : > > > On Thu, Jul 9, 2020 at 2:24 PM Tim Düsterhus wrote: > >> Ryan, >> >> Am 09.07.20 um 20:34 schrieb Ryan O'Hara: >> > I'm currently packaging this for Fedora. It seems to build just fine on >> > Fedora 32 and rawhide. Is there any new build options or dependencies >> to be >> > aware of? I'm looking at the Makefile now and nothing jumps out at me. >> That >> > said, I am totally capable of missing something. >> > >> >> I've just run `git diff v2.2-dev0..v2.3-dev0 -- Makefile`. The only >> thing I'm seeing that might of of interest to you is HLUA_PREPEND_PATH / >> HLUA_PREPEND_CPATH if you plan to ship any HAProxy specific Lua >> libraries that don't make sense in the global Lua library path. >> > > Good thing we're talking about Lua, because I just noticed that rawhide > (and Fedora 33) are using Lua 5.4 and that will not work with haproxy. I'm > investigating. Worst case scenario ... I will have to disable Lua support > in F33/rawhide. > > Ryan > > >
Re: OSX builds in Travis
I attached a patch which speeds up osx builds by 10 minutes, сб, 11 июл. 2020 г. в 00:48, Илья Шипицин : > he-he, brew bundle is deprecated (does not work) > > > https://apple.stackexchange.com/questions/148454/brew-bundle-reporting-error-unknown-command-bundle > > пт, 10 июл. 2020 г. в 14:55, Илья Шипицин : > >> >> >> пт, 10 июл. 2020 г. в 14:21, Vincent Bernat : >> >>> ❦ 9 juillet 2020 13:12 +05, Илья Шипицин: >>> >>> > do you think does it make sense to use scripted brew instead of travis >>> > plugin ? >>> > >>> > if so, we can try to "brew instal blah-blah-blah || ok, we failed, >>> lets' >>> > update and install one more time" >>> >>> I have also hit the problem several time. Brew upstream says to use >>> "brew bundle" instead: >>> >>> #v+ >>> -brew install libtool libxml2 check >>> +brew bundle --file=- <<-EOS >>> +brew "libtool" >>> +brew "libxml2" >>> +brew "check" >>> +EOS >>> #v- >>> >>> This way, brew doesn't complain if a requested package is already >>> installed but not at the latest version. >>> >> >> that's nice. >> >> I'll try to >> >> 1) get rid of socat >> 2) use brew command line >> 3) switch osx to Github Actions (thus we probably will run 4 builds in >> parallel in travis and 4 builds in Github Actions) >> >> >> >>> -- >>> It is a wise father that knows his own child. >>> -- William Shakespeare, "The Merchant of Venice" >>> >> From 49063133cf8a8e8d6d35a2bda1117ea6609910fd Mon Sep 17 00:00:00 2001 From: Ilya Shipitsin Date: Sat, 11 Jul 2020 01:15:43 +0500 Subject: [PATCH] CI: travis-ci: speed up osx build by running brew scripted we used to use travis-ci brew plugin to install "socat", travis-ci brew plugin works predictable in "all update" mode. sometimes it might take 12 minutes. let us improve developer velocity by running brew from command line. It takes 2 minutes instead of 12 minutes, --- .travis.yml | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/.travis.yml b/.travis.yml index 39b52ce2c..e455c3c77 100644 --- a/.travis.yml +++ b/.travis.yml @@ -22,9 +22,6 @@ addons: apt: update: true packages: [ liblua5.3-dev, libsystemd-dev, libpcre2-dev, clang-9, socat, ninja-build ] - homebrew: -update: true -packages: [ socat ] cache: directories: @@ -96,6 +93,8 @@ matrix: - os: osx if: type == push compiler: clang +before_script: + - HOMEBREW_NO_AUTO_UPDATE=1 brew install socat env: TARGET=osx FLAGS="USE_OPENSSL=1" OPENSSL_VERSION=1.1.1f - os: linux if: type == cron -- 2.26.2
Re: OSX builds in Travis
he-he, brew bundle is deprecated (does not work) https://apple.stackexchange.com/questions/148454/brew-bundle-reporting-error-unknown-command-bundle пт, 10 июл. 2020 г. в 14:55, Илья Шипицин : > > > пт, 10 июл. 2020 г. в 14:21, Vincent Bernat : > >> ❦ 9 juillet 2020 13:12 +05, Илья Шипицин: >> >> > do you think does it make sense to use scripted brew instead of travis >> > plugin ? >> > >> > if so, we can try to "brew instal blah-blah-blah || ok, we failed, lets' >> > update and install one more time" >> >> I have also hit the problem several time. Brew upstream says to use >> "brew bundle" instead: >> >> #v+ >> -brew install libtool libxml2 check >> +brew bundle --file=- <<-EOS >> +brew "libtool" >> +brew "libxml2" >> +brew "check" >> +EOS >> #v- >> >> This way, brew doesn't complain if a requested package is already >> installed but not at the latest version. >> > > that's nice. > > I'll try to > > 1) get rid of socat > 2) use brew command line > 3) switch osx to Github Actions (thus we probably will run 4 builds in > parallel in travis and 4 builds in Github Actions) > > > >> -- >> It is a wise father that knows his own child. >> -- William Shakespeare, "The Merchant of Venice" >> >
proposing a haproxy 2.0.16 release (was [BUG] haproxy retries dispatch to wrong server)
Hello,
On Fri, 10 Jul 2020 at 08:08, Christopher Faulet wrote:
> Hi,
>
> I finally pushed this fix in the 2.0. Note the same bug affected the HTTP
> proxy
> mode (using http_proxy option). In this case, the connection retries is now
> disabled (on the 2.0 only) because the destination address is definitely lost.
> It was the easiest way to work around the bug without backporting a bunch of
> sensitive patches from the 2.1.
Given the importance and impact of this bug you just fixed (at least 5
independent people already reported it on GH and ML) and the amount of
other important fixes already in the tree (including at least one
crash fix), I'm suggesting to release 2.0.16. Unless there are other
important open bugs with ongoing troubleshooting?
lukas@dev:~/haproxy-2.0$ git log --oneline v2.0.15..
d982a8e BUG/MEDIUM: stream-int: Disable connection retries on plain
HTTP proxy mode
e8d2423 BUG/MAJOR: stream: Mark the server address as unset on new
outgoing connection
b1e9407 MINOR: http: Add support for http 413 status
c3db7c1 BUG/MINOR: backend: Remove CO_FL_SESS_IDLE if a client remains
on the last server
0d881b2 BUG/MEDIUM: connection: Continue to recv data to a pipe when
the FD is not ready
847271d MINOR: connection: move the CO_FL_WAIT_ROOM cleanup to the reader only
39bb227 BUG/MEDIUM: mux-h1: Subscribe rather than waking up in h1_rcv_buf()
e0ca6ad BUG/MEDIUM: mux-h1: Disable splicing for the conn-stream if
read0 is received
0528ae2 BUG/MINOR: mux-h1: Disable splicing only if input data was processed
8e8168a BUG/MINOR: mux-h1: Don't read data from a pipe if the mux is
unable to receive
afadc9a BUG/MINOR: mux-h1: Fix the splicing in TUNNEL mode
8e4e357 BUG/MINOR: http_act: don't check capture id in backend (2)
c55e3e1 DOC: configuration: fix alphabetical ordering for
tune.pool-{high,low}-fd-ratio
a5e11c0 DOC: configuration: add missing index entries for
tune.pool-{low,high}-fd-ratio
ab06f88 BUG/MINOR: proxy: always initialize the trash in show servers state
ca212e5 BUG/MINOR: proxy: fix dump_server_state()'s misuse of the trash
135899e BUG/MEDIUM: pattern: Add a trailing \0 to match strings only if possible
0b77c18 DOC: ssl: add "allow-0rtt" and "ciphersuites" in crt-list
4271c77 MINOR: cli: make "show sess" stop at the last known session
8ba978b BUG/MEDIUM: fetch: Fix hdr_ip misparsing IPv4 addresses due to
missing NUL
9bd736c REGTEST: ssl: add some ssl_c_* sample fetches test
15080cb REGTEST: ssl: tests the ssl_f_* sample fetches
d6cd2b3 MINOR: spoe: Don't systematically create new applets if
processing rate is low
1b4cc2e BUG/MINOR: http_ana: clarify connection pointer check on L7 retry
d995d5f BUG/MINOR: spoe: correction of setting bits for analyzer
26e1841 REGTEST: Add a simple script to tests errorfile directives in
proxy sections
8645299 BUG/MINOR: systemd: Wait for network to be online
b88a37c MEDIUM: map: make the "clear map" operation yield
c5034a3 REGTEST: http-rules: test spaces in ACLs with master CLI
4cdff8b REGTEST: http-rules: test spaces in ACLs
c3a2e35 BUG/MINOR: mworker/cli: fix semicolon escaping in master CLI
da9a2d1 BUG/MINOR: mworker/cli: fix the escaping in the master CLI
7ed43aa BUG/MINOR: cli: allow space escaping on the CLI
249346d BUG/MINOR: spoe: add missing key length check before checking key names
1b7f58f BUG/MEDIUM: ebtree: use a byte-per-byte memcmp() to compare
memory blocks
47a5600 BUG/MINOR: tcp-rules: tcp-response must check the buffer's fullness
9f3bda0 MINOR: http: Add 404 to http-request deny
c09f797 MINOR: http: Add 410 to http-request deny
lukas@dev:~/haproxy-2.0$
Thanks,
Lukas
Re: [ANNOUNCE] haproxy-2.2.0
On Fri, Jul 10, 2020 at 11:17:33AM -0500, Ryan O'Hara wrote: > On Thu, Jul 9, 2020 at 2:24 PM Tim Düsterhus wrote: > > > Ryan, > > > > Am 09.07.20 um 20:34 schrieb Ryan O'Hara: > > > I'm currently packaging this for Fedora. It seems to build just fine on > > > Fedora 32 and rawhide. Is there any new build options or dependencies to > > be > > > aware of? I'm looking at the Makefile now and nothing jumps out at me. > > That > > > said, I am totally capable of missing something. > > > > > > > I've just run `git diff v2.2-dev0..v2.3-dev0 -- Makefile`. The only > > thing I'm seeing that might of of interest to you is HLUA_PREPEND_PATH / > > HLUA_PREPEND_CPATH if you plan to ship any HAProxy specific Lua > > libraries that don't make sense in the global Lua library path. > > > > Good thing we're talking about Lua, because I just noticed that rawhide > (and Fedora 33) are using Lua 5.4 and that will not work with haproxy. I'm > investigating. Worst case scenario ... I will have to disable Lua support > in F33/rawhide. Yep it's been reported recently, it's github issue #730. At first glance based on the errors there are some API changes that we don't know the extents yet. At least one function needs an extra argument and one enum value doesn't exist anymore from what I remember. For now it seems noone has investigated this any further yet. I hope the upgrade will be easy enough so that we can backport to 2.2. Willy
Re: [ANNOUNCE] haproxy-2.2.0
On Thu, Jul 9, 2020 at 2:24 PM Tim Düsterhus wrote: > Ryan, > > Am 09.07.20 um 20:34 schrieb Ryan O'Hara: > > I'm currently packaging this for Fedora. It seems to build just fine on > > Fedora 32 and rawhide. Is there any new build options or dependencies to > be > > aware of? I'm looking at the Makefile now and nothing jumps out at me. > That > > said, I am totally capable of missing something. > > > > I've just run `git diff v2.2-dev0..v2.3-dev0 -- Makefile`. The only > thing I'm seeing that might of of interest to you is HLUA_PREPEND_PATH / > HLUA_PREPEND_CPATH if you plan to ship any HAProxy specific Lua > libraries that don't make sense in the global Lua library path. > Good thing we're talking about Lua, because I just noticed that rawhide (and Fedora 33) are using Lua 5.4 and that will not work with haproxy. I'm investigating. Worst case scenario ... I will have to disable Lua support in F33/rawhide. Ryan
Load testing HAProxy 2.2 on x86_64 and aarch64 VMs
Hello HAProxy community, I wanted to compare how the newly released HAProxy 2.2 (Congrats!) behaves under heavy load so I've ran some tests on my x86_64 and aarch64 VMs: https://medium.com/@martin.grigorov/compare-haproxy-performance-on-x86-64-and-arm64-cpu-architectures-bfd55d1d5566 Without much surprise the x86_64 VM gave better results! It is *not* a real use case scenario: the backends serve on GET / and return "Hello World", without any file/network operations. What is interesting though is that I can get 120-160K reqs/sec when hitting directly one of the backend servers, and only 20-40K reqs/sec when using HAProxy as a load balancer in front of them. I'd be happy to re-run the tests with any kind of improvements you may have! Regards, Martin
Re: [PATCH 2/2] SMALL: ssl: Support SAN extension for certificate generation
Hello,
On Sun, Jul 05, 2020 at 09:43:23AM +0300, gers...@gmail.com wrote:
>
> Subject: Re: [PATCH 2/2] SMALL: ssl: Support SAN extension for certificate
> generation
We commonly use the 'MINOR' tag instead of 'SMALL' here.
> The use of Common Name is fading out in favor of the RFC recommended
> way of using SAN extensions. For example, Chrome from version 58
> will only match server name against SAN.
>
> The following patch adds an optional flag to attach SAN extension
> of type DNS to the generated certificate based on the server name.
> ---
> doc/configuration.txt| 8 ++
> include/haproxy/listener-t.h | 1 +
> src/cfgparse-ssl.c | 13 +
> src/ssl_sock.c | 53
> 4 files changed, 75 insertions(+)
>
> diff --git a/doc/configuration.txt b/doc/configuration.txt
> index 1d3878bc1..9a7ae43f0 100644
> --- a/doc/configuration.txt
> +++ b/doc/configuration.txt
> @@ -12166,6 +12166,14 @@ ca-sign-use-chain
>Enabling this flag will attach all public certificates encoded in
> `ca-sign-file`
>to the served certificate to the client, enabling trust.
>
> +ca-sign-use-san
> + This setting is only available when support for OpenSSL was built in. It is
> + the CA private key passphrase. This setting is optional and used only when
> + the dynamic generation of certificates is enabled. See
> + 'generate-certificates' for details.
> + Enabling this flag will add SAN extenstion of type DNS with the requested
> server name
> + inside the generated certificate.
> +
As your other patch I think that should be the default here, I don't
think we need an option for this, I'm even suprised it wasn't working
this way in the first place.
> ca-verify-file
>This setting designates a PEM file from which to load CA certificates used
> to
>verify client's certificate. It designates CA certificates which must not
> be
> diff --git a/include/haproxy/listener-t.h b/include/haproxy/listener-t.h
> index 38ca2839f..47524ffd9 100644
> --- a/include/haproxy/listener-t.h
> +++ b/include/haproxy/listener-t.h
> @@ -164,6 +164,7 @@ struct bind_conf {
> char *ca_sign_pass;/* CAKey passphrase */
>
> int ca_sign_use_chain; /* Optionally attached the certificate chain
> to the served certificate */
> + int ca_sign_use_san; /* Optionally add SAN entry to the generated
> certificate */
> struct cert_key_and_chain * ca_sign_ckch; /* CA and possible
> certificate chain for ca generation */
> #endif
> struct proxy *frontend;/* the frontend all these listeners belong
> to, or NULL */
> diff --git a/src/cfgparse-ssl.c b/src/cfgparse-ssl.c
> index 270c857f9..62f754522 100644
> --- a/src/cfgparse-ssl.c
> +++ b/src/cfgparse-ssl.c
> @@ -550,6 +550,18 @@ static int bind_parse_ca_sign_use_chain(char **args, int
> cur_arg, struct proxy *
> return 0;
> }
>
> +/* parse the "ca-sign-use-san" bind keyword */
> +static int bind_parse_ca_sign_use_san(char **args, int cur_arg, struct proxy
> *px, struct bind_conf *conf, char **err)
> +{
> +#if (defined SSL_CTRL_SET_TLSEXT_HOSTNAME && !defined
> SSL_NO_GENERATE_CERTIFICATES)
> + conf->ca_sign_use_san = 1;
> +#else
> + memprintf(err, "%sthis version of openssl cannot generate SSL
> certificates.\n",
> + err && *err ? *err : "");
> +#endif
> + return 0;
> +}
> +
> /* parse the "ca-sign-pass" bind keyword */
> static int bind_parse_ca_sign_pass(char **args, int cur_arg, struct proxy
> *px, struct bind_conf *conf, char **err)
> {
> @@ -1721,6 +1733,7 @@ static struct bind_kw_list bind_kws = { "SSL", { }, {
> { "ca-sign-file", bind_parse_ca_sign_file, 1 }, /* set
> CAFile used to generate and sign server certs */
> { "ca-sign-pass", bind_parse_ca_sign_pass, 1 }, /* set
> CAKey passphrase */
> { "ca-sign-use-chain", bind_parse_ca_sign_use_chain, 1 }, /*
> enable attaching ca chain to generated certificate */
> + { "ca-sign-use-san", bind_parse_ca_sign_use_san,1 }, /*
> enable adding SAN extension to generated certificate */
> { "ciphers", bind_parse_ciphers,1 }, /* set
> SSL cipher suite */
> #if (HA_OPENSSL_VERSION_NUMBER >= 0x10101000L)
> { "ciphersuites", bind_parse_ciphersuites, 1 }, /* set
> TLS 1.3 cipher suite */
> diff --git a/src/ssl_sock.c b/src/ssl_sock.c
> index 54829eb98..a16635341 100644
> --- a/src/ssl_sock.c
> +++ b/src/ssl_sock.c
> @@ -1745,6 +1745,54 @@ static int ssl_sock_advertise_alpn_protos(SSL *s,
> const unsigned char **out,
> #ifdef SSL_CTRL_SET_TLSEXT_HOSTNAME
> #ifndef SSL_NO_GENERATE_CERTIFICATES
>
> +int ssl_sock_add_san_ext(X509V3_CTX* ctx, X509* cert, const char
> *servername) {
A commentary on top of the function is welcomed here, to explain a
little bit what it does!
> + int failure = 0;
> + X509_EXTENSION *san_ext = NULL;
> + char *s
Re: [PATCH 1/2] MEDIUM: ssl: Support certificate chaining for certificate generation
Hello, On Sun, Jul 05, 2020 at 09:43:22AM +0300, gers...@gmail.com wrote: > From: Shimi Gersner > > haproxy supports generating SSL certificates based on SNI using a provided > CA signing certificate. Because CA certificates may be signed by multiple > CAs, in some scenarios, it is neccesary for the server to attach the trust > chain > in addition to the generated certificate. > > The following patch adds the ability to optionally serve all public > certificates provided in the `ca-sign-file` PEM file. > Certificate loading was ported to use `ca_sign_use_chain` structure, > instead of directly reading public/private keys. Totally make sense in my opinion. But if I understand correctly you only need the certificate to be signed by the leaf CA in the chain and provides all the chain to the client. We probably don't need a new option for this. So what I suggest is to put the chain in the "ca-sign-file" so it will works the same way as the "crt" keyword. > --- > doc/configuration.txt| 8 +++ > include/haproxy/listener-t.h | 4 +- > src/cfgparse-ssl.c | 13 + > src/ssl_sock.c | 98 > 4 files changed, 78 insertions(+), 45 deletions(-) > > diff --git a/doc/configuration.txt b/doc/configuration.txt > index 6d472134e..1d3878bc1 100644 > --- a/doc/configuration.txt > +++ b/doc/configuration.txt > @@ -12158,6 +12158,14 @@ ca-sign-pass >the dynamic generation of certificates is enabled. See >'generate-certificates' for details. > > +ca-sign-use-chain > + This setting is only available when support for OpenSSL was built in. It is > + the CA private key passphrase. This setting is optional and used only when Copy-paste error there :-) > + the dynamic generation of certificates is enabled. See > + 'generate-certificates' for details. > + Enabling this flag will attach all public certificates encoded in > `ca-sign-file` > + to the served certificate to the client, enabling trust. > + -- William Lallemand
Re: OSX builds in Travis
пт, 10 июл. 2020 г. в 14:21, Vincent Bernat : > ❦ 9 juillet 2020 13:12 +05, Илья Шипицин: > > > do you think does it make sense to use scripted brew instead of travis > > plugin ? > > > > if so, we can try to "brew instal blah-blah-blah || ok, we failed, lets' > > update and install one more time" > > I have also hit the problem several time. Brew upstream says to use > "brew bundle" instead: > > #v+ > -brew install libtool libxml2 check > +brew bundle --file=- <<-EOS > +brew "libtool" > +brew "libxml2" > +brew "check" > +EOS > #v- > > This way, brew doesn't complain if a requested package is already > installed but not at the latest version. > that's nice. I'll try to 1) get rid of socat 2) use brew command line 3) switch osx to Github Actions (thus we probably will run 4 builds in parallel in travis and 4 builds in Github Actions) > -- > It is a wise father that knows his own child. > -- William Shakespeare, "The Merchant of Venice" >
Re: OSX builds in Travis
❦ 9 juillet 2020 13:12 +05, Илья Шипицин: > do you think does it make sense to use scripted brew instead of travis > plugin ? > > if so, we can try to "brew instal blah-blah-blah || ok, we failed, lets' > update and install one more time" I have also hit the problem several time. Brew upstream says to use "brew bundle" instead: #v+ -brew install libtool libxml2 check +brew bundle --file=- <<-EOS +brew "libtool" +brew "libxml2" +brew "check" +EOS #v- This way, brew doesn't complain if a requested package is already installed but not at the latest version. -- It is a wise father that knows his own child. -- William Shakespeare, "The Merchant of Venice"
