Re: HAProxy and musl (was: Re: HAproxy Error)
Hi. Resuscitate this old thread with a musl lib update. https://musl.libc.org/releases.html ``` musl-1.2.4.tar.gz (sig) - May 1, 2023 This release adds TCP fallback to the DNS stub resolver, fixing the longstanding inability to query large DNS records and incompatibility with recursive nameservers that don't give partial results in truncated UDP responses. It also makes a number of other bug fixes and improvements in DNS and related functionality, including making both the modern and legacy API results differentiate between NODATA and NxDomain conditions so that the caller can handle them differently. ``` Regards Alex On 2020-04-16 (Do.) 13:26, Willy Tarreau wrote: On Thu, Apr 16, 2020 at 12:29:42PM +0200, Tim Düsterhus wrote: FWIW musl seems to work OK here when building for linux-glibc-legacy. Yes. HAProxy linked against Musl is smoke tested as part of the Docker Official Images program, because the Alpine-based Docker images use Musl as their libc. In fact you can even use TARGET=linux-glibc + USE_BACKTRACE=. By the way, I initially thought I was the only one building with musl for my EdgeRouter-x that I'm using as a distcc load balancer for the build farm at work. But if there are other users, we'd rather add a linux-musl target, as the split between OS and library was precisely made for this purpose! Anyone objects against something like this (+ the appropriate entries in other places and doc) ? diff --git a/Makefile b/Makefile index d5841a5..a3dad36 100644 --- a/Makefile +++ b/Makefile @@ -341,6 +341,18 @@ ifeq ($(TARGET),linux-glibc-legacy) USE_ACCEPT4 USE_LINUX_SPLICE USE_PRCTL USE_THREAD_DUMP USE_GETADDRINFO) endif +# For linux >= 2.6.28 and musl +ifeq ($(TARGET),linux-musl) + set_target_defaults = $(call default_opts, \ +USE_POLL USE_TPROXY USE_LIBCRYPT USE_DL USE_RT USE_CRYPT_H USE_NETFILTER \ +USE_CPU_AFFINITY USE_THREAD USE_EPOLL USE_FUTEX USE_LINUX_TPROXY \ +USE_ACCEPT4 USE_LINUX_SPLICE USE_PRCTL USE_THREAD_DUMP USE_NS USE_TFO \ +USE_GETADDRINFO) +ifneq ($(shell echo __arm__/__aarch64__ | $(CC) -E -xc - | grep '^[^\#]'),__arm__/__aarch64__) + TARGET_LDFLAGS=-latomic +endif +endif + # Solaris 8 and above ifeq ($(TARGET),solaris) # We also enable getaddrinfo() which works since solaris 8. Willy
Re: HAProxy and musl (was: Re: HAproxy Error)
On Thu, Apr 16, 2020 at 05:38:59PM +0500, ??? wrote: > ??, 16 ???. 2020 ?. ? 16:26, Willy Tarreau : > > > On Thu, Apr 16, 2020 at 12:29:42PM +0200, Tim Düsterhus wrote: > > > > FWIW musl seems to work OK here when building for linux-glibc-legacy. > > > > > > Yes. HAProxy linked against Musl is smoke tested as part of the Docker > > > Official Images program, because the Alpine-based Docker images use Musl > > > as their libc. In fact you can even use TARGET=linux-glibc + > > USE_BACKTRACE=. > > > > By the way, I initially thought I was the only one building with musl > > for my EdgeRouter-x that I'm using as a distcc load balancer for the > > build farm at work. But if there are other users, we'd rather add > > > > private buildfarm ? what do you use ? just curious Oh it's made of distccd running on 10 MIQI boards (ARM RK3288 quad-A17 at 1.8-2.0 GHz) and it's also squatting 7 of our traffic generators running on SolidRun MacchiatoBin (quad A72 at 2 GHZ) when they're not used for benchmarks. That's a lot of CPUs to deal with, so it requires a huge amount of parallelism to keep them busy. But sending too many jobs to a slow CPU or to the same machine results in uneven distribution and highly varying build times. So instead of sending everything from the PC directly to the nodes, there's a small server in the middle that runs haproxy which parses the distcc requests to figure the file sizes, and uses that value with Patrick's priority-based queuing so that largest files are distributed first to largest and most idle nodes, and small files are distributed last. This is also one of the reasons why I reorder the makefile before each release. I build with "make -j 120" and depending on the build options it takes from 3 to 5 seconds at -O0 or about 7-9 at -O2. Pretty appreciable savings when you build 50 to 200 times a day :-) There are more info there if you're interested in the design and want some photos: https://wtarreau.blogspot.com/2019/01/build-farm-version-3-2018.html https://wtarreau.blogspot.com/2019/01/build-farm-version-2-2016-2017.html I presented it twice at kernel recipes (initial one and updates) if you want to go deeper and laugh hearing me speak english with a french accent: https://kernel-recipes.org/en/2016/talks/speeding-up-development-by-setting-up-a-kernel-build-farm/ https://www.slideshare.net/ennael/kernel-recipes-2017-build-farm-again-willy-tarreau https://www.youtube.com/watch?v=sJwMRA34_SI > > a linux-musl target, as the split between OS and library was precisely > > made for this purpose! > > > > Anyone objects against something like this (+ the appropriate entries > > in other places and doc) ? > > > > looks good OK, then I'll merge it before emitting a new dev now. Thanks, Willy
Re: HAProxy and musl (was: Re: HAproxy Error)
чт, 16 апр. 2020 г. в 16:26, Willy Tarreau : > On Thu, Apr 16, 2020 at 12:29:42PM +0200, Tim Düsterhus wrote: > > > FWIW musl seems to work OK here when building for linux-glibc-legacy. > > > > Yes. HAProxy linked against Musl is smoke tested as part of the Docker > > Official Images program, because the Alpine-based Docker images use Musl > > as their libc. In fact you can even use TARGET=linux-glibc + > USE_BACKTRACE=. > > By the way, I initially thought I was the only one building with musl > for my EdgeRouter-x that I'm using as a distcc load balancer for the > build farm at work. But if there are other users, we'd rather add > private buildfarm ? what do you use ? just curious a linux-musl target, as the split between OS and library was precisely > made for this purpose! > > Anyone objects against something like this (+ the appropriate entries > in other places and doc) ? > looks good > > > diff --git a/Makefile b/Makefile > index d5841a5..a3dad36 100644 > --- a/Makefile > +++ b/Makefile > @@ -341,6 +341,18 @@ ifeq ($(TARGET),linux-glibc-legacy) > USE_ACCEPT4 USE_LINUX_SPLICE USE_PRCTL USE_THREAD_DUMP > USE_GETADDRINFO) > endif > > +# For linux >= 2.6.28 and musl > +ifeq ($(TARGET),linux-musl) > + set_target_defaults = $(call default_opts, \ > +USE_POLL USE_TPROXY USE_LIBCRYPT USE_DL USE_RT USE_CRYPT_H > USE_NETFILTER \ > +USE_CPU_AFFINITY USE_THREAD USE_EPOLL USE_FUTEX USE_LINUX_TPROXY > \ > +USE_ACCEPT4 USE_LINUX_SPLICE USE_PRCTL USE_THREAD_DUMP USE_NS > USE_TFO \ > +USE_GETADDRINFO) > +ifneq ($(shell echo __arm__/__aarch64__ | $(CC) -E -xc - | grep > '^[^\#]'),__arm__/__aarch64__) > + TARGET_LDFLAGS=-latomic > +endif > +endif > + > # Solaris 8 and above > ifeq ($(TARGET),solaris) ># We also enable getaddrinfo() which works since solaris 8. > > Willy >
Re: HAProxy and musl (was: Re: HAproxy Error)
On Thu, Apr 16, 2020 at 12:29:42PM +0200, Tim Düsterhus wrote: > > FWIW musl seems to work OK here when building for linux-glibc-legacy. > > Yes. HAProxy linked against Musl is smoke tested as part of the Docker > Official Images program, because the Alpine-based Docker images use Musl > as their libc. In fact you can even use TARGET=linux-glibc + USE_BACKTRACE=. By the way, I initially thought I was the only one building with musl for my EdgeRouter-x that I'm using as a distcc load balancer for the build farm at work. But if there are other users, we'd rather add a linux-musl target, as the split between OS and library was precisely made for this purpose! Anyone objects against something like this (+ the appropriate entries in other places and doc) ? diff --git a/Makefile b/Makefile index d5841a5..a3dad36 100644 --- a/Makefile +++ b/Makefile @@ -341,6 +341,18 @@ ifeq ($(TARGET),linux-glibc-legacy) USE_ACCEPT4 USE_LINUX_SPLICE USE_PRCTL USE_THREAD_DUMP USE_GETADDRINFO) endif +# For linux >= 2.6.28 and musl +ifeq ($(TARGET),linux-musl) + set_target_defaults = $(call default_opts, \ +USE_POLL USE_TPROXY USE_LIBCRYPT USE_DL USE_RT USE_CRYPT_H USE_NETFILTER \ +USE_CPU_AFFINITY USE_THREAD USE_EPOLL USE_FUTEX USE_LINUX_TPROXY \ +USE_ACCEPT4 USE_LINUX_SPLICE USE_PRCTL USE_THREAD_DUMP USE_NS USE_TFO \ +USE_GETADDRINFO) +ifneq ($(shell echo __arm__/__aarch64__ | $(CC) -E -xc - | grep '^[^\#]'),__arm__/__aarch64__) + TARGET_LDFLAGS=-latomic +endif +endif + # Solaris 8 and above ifeq ($(TARGET),solaris) # We also enable getaddrinfo() which works since solaris 8. Willy
Re: HAProxy and musl (was: Re: HAproxy Error)
On Thu, Apr 16, 2020 at 12:37:44PM +0200, Tim Düsterhus wrote: > Ilya, > > Am 16.04.20 um 12:34 schrieb ???: > > yep, I thought about alpine as well. > > > > I'm not sure how often official docker validation runs. If it runs often > > enough, maybe we do not need CI. > > > > The tests are run for every update of the Dockerfile, thus either for > new HAProxy releases or for Alpine updates. I usually add a `-rc` > version of the Dockerfile for upcoming HAProxy versions once the > development cycle gets close to the end and the versions are expected to > be somewhat stable. Great, I think it's already quite sufficient. Of course we can always do more and better, but we have to stay reasonable and focus on the sweet spot between the benefits expected from extra test coverage and the infrastructure costs (because at the end of the day someone *has* to pay for all the builds and tests we provoke). I think the CI has brought a lot since it was set up, and our usage is well balanced. Maybe some would consider that it's underexploited, or others woud consider that we're already abusing. My view is that it's OK this way and already helps a lot. I'm more interested by the nasty bugs it allows us to detect (such as non-portable openssl functions or incorrect integer operations on some archs) than just "it fails to build there, we need to add an ifdef", because ultimately even if we break an OS that way, it's instantly detected and trivial to fix. It's always pleasant to detect those in advance, but if they're missed it really does not harm. Detecting the issues around abns seamless on ppc64le was way more useful :-) Cheers, Willy
Re: HAProxy and musl (was: Re: HAproxy Error)
Ilya, Am 16.04.20 um 12:34 schrieb Илья Шипицин: > yep, I thought about alpine as well. > > I'm not sure how often official docker validation runs. If it runs often > enough, maybe we do not need CI. > The tests are run for every update of the Dockerfile, thus either for new HAProxy releases or for Alpine updates. I usually add a `-rc` version of the Dockerfile for upcoming HAProxy versions once the development cycle gets close to the end and the versions are expected to be somewhat stable. See here for the PRs updating the Dockerfile: https://github.com/docker-library/official-images/pulls?q=is%3Apr+label%3Alibrary%2Fhaproxy+is%3Aclosed. Within the PRs there's always a comment with the results of the tests for all the versions. Best regards Tim Düsterhus
Re: HAProxy and musl (was: Re: HAproxy Error)
yep, I thought about alpine as well. I'm not sure how often official docker validation runs. If it runs often enough, maybe we do not need CI. чт, 16 апр. 2020 г. в 15:29, Tim Düsterhus : > Willy, > > [removed Bindushree from Cc as we disgress from the main topic] > > Am 16.04.20 um 11:44 schrieb Willy Tarreau: > >> seems, we need some musl or picolibc in CI. > >> beeing glibc dependent is dangerous > > > > It's not really glibc-dependent in that it's properly enclosed in ifdefs. > > But you'd be welcome to add musl if you find an easy way to do it. Please > > just avoid building a toolchain and musl on the fly for this, as this > would > > eat a huge amount of resources on the CI infrastructure :-/ > > > > FWIW musl seems to work OK here when building for linux-glibc-legacy. > > Yes. HAProxy linked against Musl is smoke tested as part of the Docker > Official Images program, because the Alpine-based Docker images use Musl > as their libc. In fact you can even use TARGET=linux-glibc + > USE_BACKTRACE=. > > Make options are here: > > https://github.com/docker-library/haproxy/blob/3dd3917d3a70c230d8b192541ee08764e1da16af/2.2-rc/alpine/Dockerfile#L31-L45 > > Basic smoke test (Reverse Proxy to example.com) is here: > > https://github.com/docker-library/official-images/tree/master/test/tests/haproxy-basics > > Best regards > Tim Düsterhus >
HAProxy and musl (was: Re: HAproxy Error)
Willy, [removed Bindushree from Cc as we disgress from the main topic] Am 16.04.20 um 11:44 schrieb Willy Tarreau: >> seems, we need some musl or picolibc in CI. >> beeing glibc dependent is dangerous > > It's not really glibc-dependent in that it's properly enclosed in ifdefs. > But you'd be welcome to add musl if you find an easy way to do it. Please > just avoid building a toolchain and musl on the fly for this, as this would > eat a huge amount of resources on the CI infrastructure :-/ > > FWIW musl seems to work OK here when building for linux-glibc-legacy. Yes. HAProxy linked against Musl is smoke tested as part of the Docker Official Images program, because the Alpine-based Docker images use Musl as their libc. In fact you can even use TARGET=linux-glibc + USE_BACKTRACE=. Make options are here: https://github.com/docker-library/haproxy/blob/3dd3917d3a70c230d8b192541ee08764e1da16af/2.2-rc/alpine/Dockerfile#L31-L45 Basic smoke test (Reverse Proxy to example.com) is here: https://github.com/docker-library/official-images/tree/master/test/tests/haproxy-basics Best regards Tim Düsterhus
