Re: Disabling regtests in Travis ?

[Илья Шипицин]

вт, 28 янв. 2020 г. в 18:15, Willy Tarreau :

> On Tue, Jan 28, 2020 at 06:02:16PM +0500,  ??? wrote:
> > ??, 28 ???. 2020 ?. ? 16:02, Willy Tarreau :
> >
> > > On Tue, Jan 28, 2020 at 02:13:15PM +0500,  ??? wrote:
> > > > btw, we can remove "allowed failure" since this test is slow and
> excluded
> > > >
> > > > https://travis-ci.com/haproxy/haproxy/jobs/280882138
> > >
> > > My understanding of the commit message was that it was excluded because
> > > of issue #429. Am I wrong ?
> > >
> >
> > it was excluded 2 times.
> >
> > first, we marked openssl-1.0.2 as "allowed failure" because of #429
> > second, it was excluded, because it is "slow" test
>
> Hmmm no, we instead disabled slow regtests, but I'm not seeing anything
>

yep.

Skip reg-tests/ssl/set_ssl_cert.vtc because its type 'slow' is excluded



> specific there under the allow_failures section :
>
>   allow_failures:
>   - os: linux
> arch: ppc64le
> if: type == cron
> compiler: gcc
> env: TARGET=linux-glibc OPENSSL_VERSION=1.0.2u
>
> So my understanding is that it's still valid due to the error on
> "set ssl cert".
>
> Or maybe you mean something different ?
>
> Willy
>

Re: Disabling regtests in Travis ?

[Илья Шипицин]

вт, 28 янв. 2020 г. в 16:02, Willy Tarreau :

> On Tue, Jan 28, 2020 at 02:13:15PM +0500,  ??? wrote:
> > btw, we can remove "allowed failure" since this test is slow and excluded
> >
> > https://travis-ci.com/haproxy/haproxy/jobs/280882138
>
> My understanding of the commit message was that it was excluded because
> of issue #429. Am I wrong ?
>

it was excluded 2 times.

first, we marked openssl-1.0.2 as "allowed failure" because of #429
second, it was excluded, because it is "slow" test


>
> Willy
>

Re: Disabling regtests in Travis ?

[Илья Шипицин]

btw, we can remove "allowed failure" since this test is slow and excluded

https://travis-ci.com/haproxy/haproxy/jobs/280882138

пн, 27 янв. 2020 г. в 17:35, Martin Grigorov :

>
>
> On Fri, Jan 24, 2020 at 6:43 PM Willy Tarreau  wrote:
>
>> On Fri, Jan 24, 2020 at 09:12:58PM +0500,  ??? wrote:
>> > >> +  - make reg-tests VTEST_PROGRAM=../vtest/vtest
>> > >> REGTESTS_TYPES=default,bug,devel
>> > >>
>> > >
>> > > let us try that.
>>
>> OK, now pushed.
>>
>> > > I will have a look at "racy" tests.
>> > > Maybe we'll enable them on Github Actions.
>> > >
>> > >
>> > the good thing about Github Actions, it is possible to attach own build
>> > agents. So, if we
>> > have dedicated hardware and we not want to depend on travis-ci
>> neighbours,
>> > it might be an option.
>>
>> That's good to know, even if I doubt we'd need it, at least it
>> opens possibilities.
>>
>
> The regtests run fine on my ARM64 VM. I run them daily.
> If HAProxy team decides to move to GitHub Actions and to use an external
> build agent for ARM64 then just ping me!
>
> Regards,
> Martin
>
>
>>
>> Willy
>>
>>

Re: [PATCH v3] BUG/MINOR: dns: allow 63 char in hostname

[Илья Шипицин]

вс, 26 янв. 2020 г. в 23:12, William Dauchy :

> On Sun, Jan 26, 2020 at 7:08 PM Илья Шипицин  wrote:
> > such things are fragile.  once fixed, they can silently break during
> further refactoring.
> > on other hand, such functions are good candidates to write unit tests.
>
> I considered it but to my knowledge, this is currently not possible
> with varnishtest, as we would need to mock a dns resolution, and make
> haproxy starts. I don't know whether there are other plans for haproxy
> tests.
>


I do not mean varnishtest here.

varnishtest is "full stack functional test", it is too expensive.

I mean lightweight unit testing, for example, cmocka.


> --
> William
>

Re: [PATCH v3] BUG/MINOR: dns: allow 63 char in hostname

[Илья Шипицин]

such things are fragile.  once fixed, they can silently break during
further refactoring.
on other hand, such functions are good candidates to write unit tests.

maybe we should consider things like cmocka ?

вс, 26 янв. 2020 г. в 22:44, William Dauchy :

> hostname were limited to 62 char, which is not RFC1035 compliant;
> - the parsing loop should stop when above max label char
> - fix len label test where d[i] was wrongly used
> - simplify the whole function to avoid using two extra char* variable
>
> this should fix github issue #387
>
> Signed-off-by: William Dauchy 
> ---
>  src/dns.c | 31 +--
>  1 file changed, 13 insertions(+), 18 deletions(-)
>
> diff --git a/src/dns.c b/src/dns.c
> index eefd8d0dc..212c55f0d 100644
> --- a/src/dns.c
> +++ b/src/dns.c
> @@ -1470,7 +1470,6 @@ int dns_str_to_dn_label(const char *str, int
> str_len, char *dn, int dn_len)
>   */
>  int dns_hostname_validation(const char *string, char **err)
>  {
> -   const char *c, *d;
> int i;
>
> if (strlen(string) > DNS_MAX_NAME_SIZE) {
> @@ -1479,36 +1478,32 @@ int dns_hostname_validation(const char *string,
> char **err)
> return 0;
> }
>
> -   c = string;
> -   while (*c) {
> -   d = c;
> -
> +   while (*string) {
> i = 0;
> -   while (*d != '.' && *d && i <= DNS_MAX_LABEL_SIZE) {
> -   i++;
> -   if (!((*d == '-') || (*d == '_') ||
> - ((*d >= 'a') && (*d <= 'z')) ||
> - ((*d >= 'A') && (*d <= 'Z')) ||
> - ((*d >= '0') && (*d <= '9' {
> +   while (*string && *string != '.' && i <
> DNS_MAX_LABEL_SIZE) {
> +   if (!(*string == '-' || *string == '_' ||
> + (*string >= 'a' && *string <= 'z') ||
> + (*string >= 'A' && *string <= 'Z') ||
> + (*string >= '0' && *string <= '9'))) {
> if (err)
> *err = DNS_INVALID_CHARACTER;
> return 0;
> }
> -   d++;
> +   i++;
> +   string++;
> }
>
> -   if ((i >= DNS_MAX_LABEL_SIZE) && (d[i] != '.')) {
> +   if (*string == '\0')
> +   break;
> +
> +   if (*string != '.' && i >= DNS_MAX_LABEL_SIZE) {
> if (err)
> *err = DNS_LABEL_TOO_LONG;
> return 0;
> }
>
> -   if (*d == '\0')
> -   goto out;
> -
> -   c = ++d;
> +   string++;
> }
> - out:
> return 1;
>  }
>
> --
> 2.24.1
>
>
>

Re: Disabling regtests in Travis ?

[Илья Шипицин]

пт, 24 янв. 2020 г. в 20:44, Илья Шипицин :

>
>
> пт, 24 янв. 2020 г. в 20:34, Willy Tarreau :
>
>> On Fri, Jan 24, 2020 at 04:31:07PM +0100, Willy Tarreau wrote:
>> > So I'm proposing that we try a last time to run with
>> > "REGTESTS_TYPES=default,bug,devel"
>>
>> That should probably give this :
>>
>> diff --git a/.travis.yml b/.travis.yml
>> index bf4b82aa98..a82c27327d 100644
>> --- a/.travis.yml
>> +++ b/.travis.yml
>> @@ -112,7 +112,7 @@ script:
>>- ./haproxy -vv
>>- if [ "${TRAVIS_OS_NAME}" = "linux" ]; then ldd haproxy; fi
>>- if [ "${TRAVIS_OS_NAME}" = "osx" ]; then otool -L haproxy; fi
>> -  - env VTEST_PROGRAM=../vtest/vtest make reg-tests
>> +  - make reg-tests VTEST_PROGRAM=../vtest/vtest
>> REGTESTS_TYPES=default,bug,devel
>>
>
> let us try that.
>
> I will have a look at "racy" tests.
> Maybe we'll enable them on Github Actions.
>
>
the good thing about Github Actions, it is possible to attach own build
agents. So, if we
have dedicated hardware and we not want to depend on travis-ci neighbours,
it might be an option.


>
>
>>
>>  after_failure:
>>- |
>>
>

Re: Disabling regtests in Travis ?

[Илья Шипицин]

пт, 24 янв. 2020 г. в 20:34, Willy Tarreau :

> On Fri, Jan 24, 2020 at 04:31:07PM +0100, Willy Tarreau wrote:
> > So I'm proposing that we try a last time to run with
> > "REGTESTS_TYPES=default,bug,devel"
>
> That should probably give this :
>
> diff --git a/.travis.yml b/.travis.yml
> index bf4b82aa98..a82c27327d 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -112,7 +112,7 @@ script:
>- ./haproxy -vv
>- if [ "${TRAVIS_OS_NAME}" = "linux" ]; then ldd haproxy; fi
>- if [ "${TRAVIS_OS_NAME}" = "osx" ]; then otool -L haproxy; fi
> -  - env VTEST_PROGRAM=../vtest/vtest make reg-tests
> +  - make reg-tests VTEST_PROGRAM=../vtest/vtest
> REGTESTS_TYPES=default,bug,devel
>

let us try that.

I will have a look at "racy" tests.
Maybe we'll enable them on Github Actions.



>
>  after_failure:
>- |
>

[PATCH] switch to clang-9 in Linux/travis-ci builds

[Илья Шипицин]

Hello,

let us use clang-9 instead of default clang-7 for linux builds.

Cheers,
Ilya Shipitsin
From e1f7c5d551e6ab0c593f633846bf338c0fed64f1 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Fri, 24 Jan 2020 11:36:41 +0500
Subject: [PATCH] BUILD: CI: use clang-9 for travis-ci builds

---
 .travis.yml | 20 ++--
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index bf4b82aa9..bba7d74a2 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -20,7 +20,7 @@ env:
 addons:
   apt:
 update: true
-packages: [ liblua5.3-dev, libsystemd-dev, libpcre2-dev, socat ]
+packages: [ liblua5.3-dev, libsystemd-dev, libpcre2-dev, clang-9, socat ]
   homebrew:
 update: true
 packages: [ socat ]
@@ -41,12 +41,12 @@ matrix:
 arch: amd64
 if: type != cron
 compiler: clang
-env: TARGET=linux-glibc OPENSSL_VERSION=1.1.1d
+env: TARGET=linux-glibc OPENSSL_VERSION=1.1.1d CC=clang-9
   - os: linux
 arch: arm64
 if: type != cron
 compiler: clang
-env: TARGET=linux-glibc OPENSSL_VERSION=1.1.1d
+env: TARGET=linux-glibc OPENSSL_VERSION=1.1.1d CC=clang-9
   - os: linux
 if: type == cron
 compiler: clang
@@ -60,27 +60,27 @@ matrix:
   - os: linux
 if: type == cron
 compiler: clang
-env: TARGET=linux-glibc OPENSSL_VERSION=1.1.0l FIFTYONEDEGREES_SRC="contrib/51d/src/trie"
+env: TARGET=linux-glibc OPENSSL_VERSION=1.1.0l FIFTYONEDEGREES_SRC="contrib/51d/src/trie" CC=clang-9
   - os: linux
 if: type != cron
 compiler: clang
-env: TARGET=linux-glibc LIBRESSL_VERSION=3.0.2
+env: TARGET=linux-glibc LIBRESSL_VERSION=3.0.2 CC=clang-9
   - os: linux
 if: type == cron
 compiler: clang
-env: TARGET=linux-glibc LIBRESSL_VERSION=2.9.2
+env: TARGET=linux-glibc LIBRESSL_VERSION=2.9.2 CC=clang-9
   - os: linux
 if: type == cron
 compiler: clang
-env: TARGET=linux-glibc LIBRESSL_VERSION=2.8.3 EXTRA_OBJS="contrib/prometheus-exporter/service-prometheus.o"
+env: TARGET=linux-glibc LIBRESSL_VERSION=2.8.3 EXTRA_OBJS="contrib/prometheus-exporter/service-prometheus.o" CC=clang-9
   - os: linux
 if: type == cron
 compiler: clang
-env: TARGET=linux-glibc BORINGSSL=yes
+env: TARGET=linux-glibc BORINGSSL=yes CC=clang-9
   - os: linux
 if: type != cron
 compiler: clang
-env: TARGET=linux-glibc FLAGS=
+env: TARGET=linux-glibc FLAGS= CC=clang-9
   - os: osx
 if: type != cron
 compiler: clang
@@ -88,7 +88,7 @@ matrix:
   - os: linux
 if: type == cron
 compiler: clang
-env: TARGET=linux-glibc FLAGS="USE_SLZ=1 USE_PCRE2=1 USE_PCRE2_JIT=1 USE_LUA=1 USE_OPENSSL=1 USE_SYSTEMD=1 USE_WURFL=1 WURFL_INC=contrib/wurfl WURFL_LIB=contrib/wurfl USE_51DEGREES=1"
+env: TARGET=linux-glibc FLAGS="USE_SLZ=1 USE_PCRE2=1 USE_PCRE2_JIT=1 USE_LUA=1 USE_OPENSSL=1 USE_SYSTEMD=1 USE_WURFL=1 WURFL_INC=contrib/wurfl WURFL_LIB=contrib/wurfl USE_51DEGREES=1" CC=clang-9
 before_script:
   - git clone http://git.1wt.eu/git/libslz.git/
   - cd libslz && make && make PREFIX=${HOME}/opt install && cd ..
-- 
2.24.1

Re: [PATCH[ re-enable cygwin CI builds (Github Actions)

[Илья Шипицин]

чт, 23 янв. 2020 г. в 15:20, Willy Tarreau :

> On Wed, Jan 22, 2020 at 11:10:05PM +0100, Tim Düsterhus wrote:
> > > both travis and github actions do offer 4 parallel builds, while
> cirrus and
> > > app veyor offer 1 parallel build.
> >
> > Parallel Builds just improve test speed. I don't consider that an
> > important selling point for us. The development process is fairly
> > asynchronous anyway and the important thing is that there are results
> > for more obscure configurations, not that there results within 1 minute.
>
> Sometimes it's nice to get a low-latency feedback on recent changes. But
> lately travis has become amazingly slow, turning from tens of seconds to
> several minutes, and regtests occasionally failing on timeouts, so it's
> clear that this low-latency argument is not true anymore. I think they're
> victim of their success and the spawned VMs are simply overloaded by too
> many users doing lots of builds there. We do try to be good citizens by
> limiting our combinations but it's possibly not the case for everyone.
>
> I still think we should not be too demanding on these tools. They do
> help us, 2.0 and 2.1's quality at the release date was far above
> anything we've done in the past, so I'm fine if we continue to use a
> diversity of tools depending on their capabilities. For sure it's less
> work to always look at a single place but that's no big deal overall.
>


we can simplify things a bit.
if we turn README into README.md , so we can add badges and links to builds.
it saves time, badges are either green or red. if it is green, ok, no need
to look deeper (amd mo need to remember links to CI)


like that

[image: Screenshot from 2020-01-24 10-59-18.png]



>
> Willy
>

Re: arm64 builds?

[Илья Шипицин]

пт, 24 янв. 2020 г. в 01:04, Илья Шипицин :

>
>
> пт, 24 янв. 2020 г. в 00:54, Willy Tarreau :
>
>> On Fri, Jan 24, 2020 at 12:46:12AM +0500,  ??? wrote:
>> > > diff --git a/Makefile b/Makefile
>> > > index 8399f6ca35..4757bc77e6 100644
>> > > --- a/Makefile
>> > > +++ b/Makefile
>> > > @@ -199,6 +199,7 @@ SPEC_CFLAGS += $(call
>> cc-opt,-Wshift-negative-value)
>> > >  SPEC_CFLAGS += $(call cc-opt,-Wshift-overflow=2)
>> > >  SPEC_CFLAGS += $(call cc-opt,-Wduplicated-cond)
>> > >  SPEC_CFLAGS += $(call cc-opt,-Wnull-dereference)
>> > > +SPEC_CFLAGS += $(call cc-opt,-Walloc-size-larger-than=-1)
>> > >
>> >
>> >
>> >   CC  src/cfgparse.o
>> > src/cfgparse.c: In function 'check_config_validity':
>> > src/cfgparse.c:3642:33: warning: product '2147483648 * 8' of arguments 1
>> > and 2 exceeds 'SIZE_MAX' [-Walloc-size-larger-than=]
>>
>> Pfff The only good news is that it takes -1 as SIZE_MAX.
>>
>> >  newsrv->idle_orphan_conns = calloc((unsigned int)global.nbthread,
>> > sizeof(*newsrv->idle_orphan_conns));
>> >
>> >
>> ^
>> > src/cfgparse.c:3642:33: note: argument 1 in the range [2147483648,
>> > 4294967295]
>> (...)
>> > why is it complaining about "product '2147483648 * 8" ?
>>
>> because calloc multiplies the two fields and gcc decided that the largest
>> value we could possibly pass to the first one if we were as stupid as it
>> is is 2147483648. Interestingly it took the largest negative value turned
>> to positive and ignored the positive ones that can be turned to the second
>> half that are negative if nbthread was negative.
>>
>> I really think this test is totally bogus and that there is no way to
>> express it correctly. I mean, gcc only lets us use 8, 16, 32 or 64 bits.
>> If you need to calloc a few megabytes, you'll be forced to apply a mask
>> to the value just to shut it up, and *create* the overflow problem
>> yourself
>> when it didn't exist.
>>
>> Let's give up on this one if it doesn't cause too much trouble to you.
>> Otherwise we might cheat doing this :
>>
>> calloc((unsigned short)global.nbthread, ...)
>>
>> But I really despise this given that we have to make the code wrong just
>> to please this shitty compiler.
>>
>
>
> it was ubuntu 18.04 + gcc8, I'll try 19.10 + gcc9
>

gcc9 produces the same warning


>
>
>>
>> Willy
>>
>

Re: arm64 builds?

[Илья Шипицин]

пт, 24 янв. 2020 г. в 00:54, Willy Tarreau :

> On Fri, Jan 24, 2020 at 12:46:12AM +0500,  ??? wrote:
> > > diff --git a/Makefile b/Makefile
> > > index 8399f6ca35..4757bc77e6 100644
> > > --- a/Makefile
> > > +++ b/Makefile
> > > @@ -199,6 +199,7 @@ SPEC_CFLAGS += $(call
> cc-opt,-Wshift-negative-value)
> > >  SPEC_CFLAGS += $(call cc-opt,-Wshift-overflow=2)
> > >  SPEC_CFLAGS += $(call cc-opt,-Wduplicated-cond)
> > >  SPEC_CFLAGS += $(call cc-opt,-Wnull-dereference)
> > > +SPEC_CFLAGS += $(call cc-opt,-Walloc-size-larger-than=-1)
> > >
> >
> >
> >   CC  src/cfgparse.o
> > src/cfgparse.c: In function 'check_config_validity':
> > src/cfgparse.c:3642:33: warning: product '2147483648 * 8' of arguments 1
> > and 2 exceeds 'SIZE_MAX' [-Walloc-size-larger-than=]
>
> Pfff The only good news is that it takes -1 as SIZE_MAX.
>
> >  newsrv->idle_orphan_conns = calloc((unsigned int)global.nbthread,
> > sizeof(*newsrv->idle_orphan_conns));
> >
> >
> ^
> > src/cfgparse.c:3642:33: note: argument 1 in the range [2147483648,
> > 4294967295]
> (...)
> > why is it complaining about "product '2147483648 * 8" ?
>
> because calloc multiplies the two fields and gcc decided that the largest
> value we could possibly pass to the first one if we were as stupid as it
> is is 2147483648. Interestingly it took the largest negative value turned
> to positive and ignored the positive ones that can be turned to the second
> half that are negative if nbthread was negative.
>
> I really think this test is totally bogus and that there is no way to
> express it correctly. I mean, gcc only lets us use 8, 16, 32 or 64 bits.
> If you need to calloc a few megabytes, you'll be forced to apply a mask
> to the value just to shut it up, and *create* the overflow problem yourself
> when it didn't exist.
>
> Let's give up on this one if it doesn't cause too much trouble to you.
> Otherwise we might cheat doing this :
>
> calloc((unsigned short)global.nbthread, ...)
>
> But I really despise this given that we have to make the code wrong just
> to please this shitty compiler.
>


it was ubuntu 18.04 + gcc8, I'll try 19.10 + gcc9


>
> Willy
>

Re: arm64 builds?

[Илья Шипицин]

чт, 23 янв. 2020 г. в 23:17, Willy Tarreau :

> On Thu, Jan 23, 2020 at 10:59:00PM +0500,  ??? wrote:
> > > Could you please try to use (size_t) instead of (unsigned int) ? If
> it's
> > > enough to shut it up, I'm fine with doing that change. Otherwise we'll
> > > probably get rid of that stupid warning.
> > >
> >
> >   CC  src/server.o
> >   CC  src/cfgparse.o
> > src/cfgparse.c: In function 'check_config_validity':
> > src/cfgparse.c:3642:33: warning: argument 1 range [2147483648,
> 4294967295]
> > exceeds maximum object size 2147483647 [-Walloc-size-larger-than=]
> >  newsrv->idle_orphan_conns = calloc((size_t)global.nbthread,
> > sizeof(*newsrv->idle_orphan_conns));
> >
> >  ^~~
> > In file included from src/cfgparse.c:24:
> > /usr/arm-linux-gnueabihf/include/stdlib.h:541:14: note: in a call to
> > allocation function 'calloc' declared here
> >  extern void *calloc (size_t __nmemb, size_t __size)
> >   ^~
> >   CC  src/checks.o
> > ^CMakefile:846: recipe for target 'src/checks.o' failed
> > make: *** [src/checks.o] Interrupt
>
> Thanks for the test! So basically this clearly proves we respect the
> calling convention but the compiler still complains. OK I'm seeing in
> the mad that it's for functions "decorated" with the "alloc_size"
> attribute. Thus in short they enforce constraints that cannot be
> expressed with available types. This is becoming totally ridiculous.
>
> We're getting a huge collection of stupid warnings to shut up. I
> suspect that the sum of all the code written to shut them is larger
> than what haproxy 1.0 used to be :-/
>
> The man page says we can disable this crap using
> -Walloc-size-larger-than=SIZE_MAX but on my compiler (8.2) it simply
> fails to build. However when passing explicit values not even that
> large, I don't get any such warning, so I'm starting to think that
> it's a real bug of GCC 9.2, because quite frankly, aside calling
> calloc() with a char or short in argument, there's almost no way out
> of this absurdity.
>
> For me, calling it with -Walloc-size-larger-than=-1 makes it stay
> silent. is it also the case for you ? You can patch your Makefile this
> way:
>
> diff --git a/Makefile b/Makefile
> index 8399f6ca35..4757bc77e6 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -199,6 +199,7 @@ SPEC_CFLAGS += $(call cc-opt,-Wshift-negative-value)
>  SPEC_CFLAGS += $(call cc-opt,-Wshift-overflow=2)
>  SPEC_CFLAGS += $(call cc-opt,-Wduplicated-cond)
>  SPEC_CFLAGS += $(call cc-opt,-Wnull-dereference)
> +SPEC_CFLAGS += $(call cc-opt,-Walloc-size-larger-than=-1)
>


  CC  src/cfgparse.o
src/cfgparse.c: In function ‘check_config_validity’:
src/cfgparse.c:3642:33: warning: product ‘2147483648 * 8’ of arguments 1
and 2 exceeds ‘SIZE_MAX’ [-Walloc-size-larger-than=]
 newsrv->idle_orphan_conns = calloc((unsigned int)global.nbthread,
sizeof(*newsrv->idle_orphan_conns));

 ^
src/cfgparse.c:3642:33: note: argument 1 in the range [2147483648,
4294967295]
In file included from src/cfgparse.c:24:
/usr/arm-linux-gnueabihf/include/stdlib.h:541:14: note: in a call to
allocation function ‘calloc’ declared here
 extern void *calloc (size_t __nmemb, size_t __size)
  ^~
  CC  src/checks.o
  CC  src/backend.o




why is it complaining about "product ‘2147483648 * 8" ?




>
>   Memory usage tuning
>  # If small memory footprint is required, you can reduce the buffer size.
> There
>
> If it still fails, we'll have to ignore it and wait for this monstrosity
> to be fixed by their authors :-/
>
> Willy
>

Re: arm64 builds?

[Илья Шипицин]

чт, 23 янв. 2020 г. в 15:14, Willy Tarreau :

> On Thu, Jan 23, 2020 at 01:09:22PM +0500,  ??? wrote:
> > hello,
> >
> > I tried to build using cross compiler (arm32 on amd64).  sorry for
> > screenshot.
> > Willy, do you mean errors like that ?
>
> So for those not seeing the screenshot, it says:
> warning: argument 1 range [2147483648, 4294967295] exceeds maximum object
> size 2147483647 :
>
>  new->idle_orphan_conns = calloc((unsigned int)global.nbthread,
> sizeof(*new->idle_orphan_conns));
>
> Thus the cast to unsigned int was likely placed there to shut it up
> once. Looking at the man page, it says:
>
>void *calloc(size_t nmemb, size_t size);
>
> size_t is unsigned (unless you're using an older gcc < 2.4 on a unusual
> OS). I don't see how it can be said to have a maximum size of 2147483647.
>
> This is exactly the type of stupid warnings that causes us to add casts
> that hide real bugs :-/
>
> Could you please try to use (size_t) instead of (unsigned int) ? If it's
> enough to shut it up, I'm fine with doing that change. Otherwise we'll
> probably get rid of that stupid warning.
>

  CC  src/server.o
  CC  src/cfgparse.o
src/cfgparse.c: In function ‘check_config_validity’:
src/cfgparse.c:3642:33: warning: argument 1 range [2147483648, 4294967295]
exceeds maximum object size 2147483647 [-Walloc-size-larger-than=]
 newsrv->idle_orphan_conns = calloc((size_t)global.nbthread,
sizeof(*newsrv->idle_orphan_conns));

 ^~~
In file included from src/cfgparse.c:24:
/usr/arm-linux-gnueabihf/include/stdlib.h:541:14: note: in a call to
allocation function ‘calloc’ declared here
 extern void *calloc (size_t __nmemb, size_t __size)
  ^~
  CC  src/checks.o
^CMakefile:846: recipe for target 'src/checks.o' failed
make: *** [src/checks.o] Interrupt



>
> Thanks!
> Willy
>

Re: [PATCH[ re-enable cygwin CI builds (Github Actions)

[Илья Шипицин]

чт, 23 янв. 2020 г. в 20:54, Willy Tarreau :

> On Thu, Jan 23, 2020 at 08:40:19PM +0500,  ??? wrote:
> > those timeouts are not related to travis itself, I beleive they are
> mostly
> > related to either real failures or tests instability (race condition).
>
> These tests are racy by nature and some rely on short delays (i.e. health
> checks). If tests are run on an overloaded machine I don't find it
> surprising that a few will fail. Often I click restart and they succeed.
>


we started to observe asan failures

***  h1   debug|AddressSanitizer:DEADLYSIGNAL

***  h1   
debug|=

***  h1   debug|==10330==ERROR: AddressSanitizer: SEGV on unknown
address 0x0018 (pc 0x006e27b1 bp 0x0003 sp
0x7ffe0d8ee080 T0)

***  h1   debug|==10330==The signal is caused by a READ memory access.

***  h1   debug|==10330==Hint: address points to the zero page.

 dT   0.015

***  h1   debug|==10330==WARNING: failed to fork (errno 11)

***  h1   debug|==10330==WARNING: failed to fork (errno 11)

***  h1   debug|==10330==WARNING: failed to fork (errno 11)

 dT   0.016

***  h1   debug|==10330==WARNING: failed to fork (errno 11)

***  h1   debug|==10330==WARNING: failed to fork (errno 11)

***  h1   debug|==10330==WARNING: Failed to use and restart external symbolizer!

***  h1   debug|#0 0x6e27b0
(/home/travis/build/haproxy/haproxy/haproxy+0x6e27b0)

***  h1   debug|#1 0x551799
(/home/travis/build/haproxy/haproxy/haproxy+0x551799)

***  h1   debug|#2 0x8218ed
(/home/travis/build/haproxy/haproxy/haproxy+0x8218ed)

***  h1   debug|#3 0x735000
(/home/travis/build/haproxy/haproxy/haproxy+0x735000)

***  h1   debug|#4 0x7328c8
(/home/travis/build/haproxy/haproxy/haproxy+0x7328c8)

***  h1   debug|#5 0x7ff91cd88b96  (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)

***  h1   debug|#6 0x41bd09
(/home/travis/build/haproxy/haproxy/haproxy+0x41bd09)

***  h1   debug|

***  h1   debug|AddressSanitizer can not provide additional info.

***  h1   debug|SUMMARY: AddressSanitizer: SEGV
(/home/travis/build/haproxy/haproxy/haproxy+0x6e27b0)

***  h1   debug|==10330==ABORTING

 dT   0.017

 c1   fd=9 EOF, as expected




I will have a look. Maybe we will switch asan off for good.



> These ones often fail there and only there, so the environment definitely
> has an impact. And seeing the execution time which has become 10-30 times
> what it is on simple laptop really makes me feel like the VMs are seriously
> stressed.
>
> > the bigger is number of tests, the more we depend on those timeouts.
> >
> > however, we can try to run tests in parallel :)
>
> It would be worse!
>
> Willy
>

Re: [PATCH[ re-enable cygwin CI builds (Github Actions)

[Илья Шипицин]

чт, 23 янв. 2020 г. в 15:20, Willy Tarreau :

> On Wed, Jan 22, 2020 at 11:10:05PM +0100, Tim Düsterhus wrote:
> > > both travis and github actions do offer 4 parallel builds, while
> cirrus and
> > > app veyor offer 1 parallel build.
> >
> > Parallel Builds just improve test speed. I don't consider that an
> > important selling point for us. The development process is fairly
> > asynchronous anyway and the important thing is that there are results
> > for more obscure configurations, not that there results within 1 minute.
>
> Sometimes it's nice to get a low-latency feedback on recent changes. But
> lately travis has become amazingly slow, turning from tens of seconds to
> several minutes, and regtests occasionally failing on timeouts, so it's
> clear that this low-latency argument is not true anymore. I think they're
> victim of their success and the spawned VMs are simply overloaded by too
> many users doing lots of builds there. We do try to be good citizens by
> limiting our combinations but it's possibly not the case for everyone.
>

those timeouts are not related to travis itself, I beleive they are mostly
related to either
real failures or tests instability (race condition).

the bigger is number of tests, the more we depend on those timeouts.

however, we can try to run tests in parallel :)


>
> I still think we should not be too demanding on these tools. They do
> help us, 2.0 and 2.1's quality at the release date was far above
> anything we've done in the past, so I'm fine if we continue to use a
> diversity of tools depending on their capabilities. For sure it's less
> work to always look at a single place but that's no big deal overall.
>
> Willy
>

Re: arm64 builds?

[Илья Шипицин]

чт, 23 янв. 2020 г. в 15:14, Willy Tarreau :

> On Thu, Jan 23, 2020 at 01:09:22PM +0500,  ??? wrote:
> > hello,
> >
> > I tried to build using cross compiler (arm32 on amd64).  sorry for
> > screenshot.
> > Willy, do you mean errors like that ?
>
> So for those not seeing the screenshot, it says:
> warning: argument 1 range [2147483648, 4294967295] exceeds maximum object
> size 2147483647 :
>
>  new->idle_orphan_conns = calloc((unsigned int)global.nbthread,
> sizeof(*new->idle_orphan_conns));
>
> Thus the cast to unsigned int was likely placed there to shut it up
> once. Looking at the man page, it says:
>
>void *calloc(size_t nmemb, size_t size);
>
> size_t is unsigned (unless you're using an older gcc < 2.4 on a unusual
> OS). I don't see how it can be said to have a maximum size of 2147483647.
>
> This is exactly the type of stupid warnings that causes us to add casts
> that hide real bugs :-/
>
> Could you please try to use (size_t) instead of (unsigned int) ? If it's
> enough to shut it up, I'm fine with doing that change. Otherwise we'll
> probably get rid of that stupid warning.
>

yep.
I'll play with it a liitle bit, and hopefully we will add cross builds to
Github Actions.


also, can we treat warnings as errors for CI builds ? it would save a bunch
of time, instead of
looking at build log, we can watch for build status.


>
> Thanks!
> Willy
>

Re: arm64 builds?

[Илья Шипицин]

hello,

I tried to build using cross compiler (arm32 on amd64).  sorry for
screenshot.
Willy, do you mean errors like that ?


[image: Screenshot from 2020-01-23 13-03-49.png]


ср, 6 нояб. 2019 г. в 12:26, Willy Tarreau :

> Hi Ilya,
>
> On Tue, Nov 05, 2019 at 07:20:43PM +0500,  ??? wrote:
> > only arm64 are available.
> > we can try arm using cross build, for example.
>
> Then don't bother with this, it's likely then that they will not
> have all the environment available. Maybe even their hardware does
> not support arm32 at all. It was just a suggestion to try to optimize
> the solution but even arm64 is already nice to have.
>
> > is it common way to use cross build for building haproxy ?
>
> Yes it is. Actually I don't think I've built it natively for a very
> long time now, as even on my laptop I'm used to stick to the cross-
> compilers which are distributed on the distcc build farm running on
> the lab load generators :-)
>
> In practice just pass "CC=/path/to/gcc" and let it do its job. It will
> automatically use LD=$(CC) if you don't override it. You may have to
> pass PCRE_INC/PCRE_LIB, SSL_INC/SSL_LIB, LUA_INC/LUA_LIB but that's
> about all.
>
> Just for reference here's the command line I'm using when building
> (and a variant with -O0 which builds in 3.5 seconds). It may look
> large because I force all debugging options but it's in a script so
> I don't have to type it :-)
>
>PATH=/f/tc/x86_64-gcc47_glibc218-linux-gnu/bin:$PATH make -j 120
> TMPDIR=/dev/shm DISTCC_FALLBACK=0 DISTCC_SKIP_LOCAL_RETRY=0
> DISTCC_BACKOFF_PERIOD=0 DISTCC_PAUSE_TIME_MSEC=50
> DISTCC_HOSTS="--localslots_cpp=120 10.0.0.235/120,lzo"
> CC=/g/public/linux/master/x86_64-gcc47_glibc218-linux-gnu-gcc
> TARGET=linux-glibc DEP= USE_PCRE=1 PCREDIR=
> DEFINE="-DDEBUG_DONT_SHARE_POOLS -DDEBUG_MEMORY_POOLS -DDEBUG_UAF
> -DDEBUG_EXPR -DDEBUG_STRICT -DDEBUG_DEV" USE_OPENSSL=1 USE_ZLIB=1 USE_LUA=1
> LUA_LIB_NAME=lua EXTRA= USE_SLZ=1 SLZ_INC=/g/public/slz/src
> SLZ_LIB=/g/public/slz USE_ZLIB= USE_DEVICEATLAS=1
> DEVICEATLAS_SRC=contrib/deviceatlas USE_51DEGREES=1
> 51DEGREES_SRC=contrib/51d/src/trie USE_WURFL=1 WURFL_INC=contrib/wurfl
> WURFL_LIB=contrib/wurfl CPU_CFLAGS="-O2" "$@"
>
> When testing with various local openssl branches I do have another variant
> of this which uses the local, native pre-processor and linkers, and remote
> compilers. It's a bit ugly since they're not on the same version but in
> practice it works fine.
>
> Cheers,
> Willy
>

Re: [PATCH[ re-enable cygwin CI builds (Github Actions)

[Илья Шипицин]

чт, 23 янв. 2020 г. в 03:10, Tim Düsterhus :

> Ilya,
>
> Am 22.01.20 um 23:04 schrieb Илья Шипицин:
> >> Yes, that's my understanding of GitHub actions as well. However I
> >> dislike having three types of CI (Travis, Cirrus and GitHub Actions).
> >> Can Travis be replaced with GitHub Actions for our use case? I guess
> >> Cirrus can't, because FreeBSD?
> >
> > both travis and github actions do offer 4 parallel builds, while cirrus
> and
> > app veyor offer 1 parallel build.
>
> Parallel Builds just improve test speed. I don't consider that an
> important selling point for us. The development process is fairly
> asynchronous anyway and the important thing is that there are results
> for more obscure configurations, not that there results within 1 minute.
> However ...
>
> > travis-ci offers ARM64, ppc64le and s390x (not available on github
> actions).
>
> ... that's a good argument to keep Travis-CI. Too bad, I like the GitHub
> Actions integration better.
>
>
me too :)

travis is not comfortable for choosing custom images (for example, if you
wish to build on Fedora or Arch).



> >>> +- name: fake step
> >>
> >> Give a proper name to that step. "Show pwd" is fine.
> >>
> >
> >
> > there should be no such step.
> > however, without that step cygwin fails for no visible reason.
>
> Then it's even more important to give a good name (or comment).
> Otherwise you might risk that someone removes it accidentally!
>
> >> 2. Split the ./haproxy -vv into a separate step, if that's possible.
> >>
> > sure, it's possible
> >
>
> Perfect. I wasn't sure whether the environment was somehow cleaned up in
> between the steps.
>
> Best regards
> Tim Düsterhus
>

Re: [PATCH[ re-enable cygwin CI builds (Github Actions)

[Илья Шипицин]

чт, 23 янв. 2020 г. в 02:59, Tim Düsterhus :

> Ilya,
>
> Am 22.01.20 um 22:41 schrieb Илья Шипицин:
> > let us move it to Github Actions.
> > as far as I tried no extra step are required for enabling, just proper
> file
> > in
> >
> > .github/workflows
>
> Yes, that's my understanding of GitHub actions as well. However I
> dislike having three types of CI (Travis, Cirrus and GitHub Actions).
> Can Travis be replaced with GitHub Actions for our use case? I guess
> Cirrus can't, because FreeBSD?
>


both travis and github actions do offer 4 parallel builds, while cirrus and
app veyor offer 1 parallel build.

travis-ci offers ARM64, ppc64le and s390x (not available on github actions).



>
> To the patch:
>
> > From 372c547a94ff02fa04ca052a87863161d3e85b37 Mon Sep 17 00:00:00 2001
> > From: Ilya Shipitsin 
> > Date: Thu, 23 Jan 2020 02:33:38 +0500
> > Subject: [PATCH] BUILD: CI: move cygwin builds to Github Actions
> >
> > builds on travis-ci fail because of
> >
> https://travis-ci.community/t/cygwin-issue-cygheap-base-mismatch-detected/5359
> > unfortunately, documentation does not help.
> >
> > so, let us move builds to Github Actions.
> >
> > also, remove deprecated "sudo" directive from .travis.yml
>
> Could you please create separate patches for several unrelated changes?
> If there's an "also" in the commit message then that's an indication
> that the patch should be split.
>
> Single-line patches are perfectly acceptable. I send series containing
> multiple single-line patches to the list on a regular basis (as distinct
> emails even, because I use git-send-email).
>
> > +steps:
> > +- uses: actions/checkout@v1
> > +- name: install prerequisites
> > +  run: choco install bash make libssl-devel cygwin-devel gcc-core
> libgcc1 binutils lua-devel libpcre-devel zlib-devel --source cygwin
> > +- name: fake step
>
> Give a proper name to that step. "Show pwd" is fine.
>


there should be no such step.
however, without that step cygwin fails for no visible reason.

I'll remove this step after I'll find it out.


>
> > +  run: C:\\tools\\cygwin\\bin\\bash -lc 'pwd'
> > +- name: build
> > +  run: C:\\tools\\cygwin\\bin\\bash -lc 'cd $OLDPWD && make
> TARGET=cygwin USE_ZLIB=1 USE_PCRE=1 USE_PCRE_JIT=1 USE_LUA=1 USE_OPENSSL=1
> USE_THREAD=1 && ./haproxy -vv'
>
> 1. Does GitHub Actions support variables for the build flags, similar to
> Travis? That would make things more readable.
>


yes.


>
> 2. Split the ./haproxy -vv into a separate step, if that's possible.
>


sure, it's possible

>
> Best regards
> Tim Düsterhus
>

[PATCH[ re-enable cygwin CI builds (Github Actions)

[Илья Шипицин]

Hello,

I spent lots of time trying to get cygwin on travis working.
no luck.

let us move it to Github Actions.
as far as I tried no extra step are required for enabling, just proper file
in

.github/workflows

(attached in patch).


Cheers,
Ilya Shipitcin
From 372c547a94ff02fa04ca052a87863161d3e85b37 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Thu, 23 Jan 2020 02:33:38 +0500
Subject: [PATCH] BUILD: CI: move cygwin builds to Github Actions

builds on travis-ci fail because of
https://travis-ci.community/t/cygwin-issue-cygheap-base-mismatch-detected/5359
unfortunately, documentation does not help.

so, let us move builds to Github Actions.

also, remove deprecated "sudo" directive from .travis.yml
---
 .github/workflows/windows-latest.yml | 20 
 .travis.yml  |  7 ---
 2 files changed, 20 insertions(+), 7 deletions(-)
 create mode 100644 .github/workflows/windows-latest.yml

diff --git a/.github/workflows/windows-latest.yml b/.github/workflows/windows-latest.yml
new file mode 100644
index 0..27b16eec6
--- /dev/null
+++ b/.github/workflows/windows-latest.yml
@@ -0,0 +1,20 @@
+# build status appears on https://github.com/haproxy/haproxy/actions
+
+name: windows-latest
+
+on: [push]
+
+jobs:
+  cygwin:
+
+runs-on: windows-latest
+
+steps:
+- uses: actions/checkout@v1
+- name: install prerequisites
+  run: choco install bash make libssl-devel cygwin-devel gcc-core libgcc1 binutils lua-devel libpcre-devel zlib-devel --source cygwin
+- name: fake step
+  run: C:\\tools\\cygwin\\bin\\bash -lc 'pwd'
+- name: build
+  run: C:\\tools\\cygwin\\bin\\bash -lc 'cd $OLDPWD && make TARGET=cygwin USE_ZLIB=1 USE_PCRE=1 USE_PCRE_JIT=1 USE_LUA=1 USE_OPENSSL=1 USE_THREAD=1 && ./haproxy -vv'
+
diff --git a/.travis.yml b/.travis.yml
index a0d9502b4..bf4b82aa9 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,6 +1,5 @@
 # build status appears on https://travis-ci.com/haproxy/haproxy
 
-sudo: required
 dist: bionic
 
 language: c
@@ -86,12 +85,6 @@ matrix:
 if: type != cron
 compiler: clang
 env: TARGET=osx FLAGS="USE_OPENSSL=1" OPENSSL_VERSION=1.1.1d
-#  - os: windows
-#if: type == cron
-#install:
-#  - choco install bash make libssl-devel cygwin-devel gcc-core libgcc1 binutils lua-devel libpcre-devel zlib-devel --source cygwin
-#script:
-#  - C:\\tools\\cygwin\\bin\\bash -lc 'cd $OLDPWD && make TARGET=cygwin USE_ZLIB=1 USE_PCRE=1 USE_PCRE_JIT=1 USE_LUA=1 USE_OPENSSL=1 USE_THREAD=1 && ./haproxy -vv'
   - os: linux
 if: type == cron
 compiler: clang
-- 
2.24.1

[PATCH] temporarily mark openssl-1.0.2 builds as allowed failure

[Илья Шипицин]

Hello,

this is a follow up patch for the recent travis-ci improvement.

Cheers,
Ilya Shipitcin
From f13c8cd8c28d376c914cee24ac9b7e09a7203473 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Wed, 22 Jan 2020 15:23:51 +0500
Subject: [PATCH] BUILD: CI: temporarily mark openssl-1.0.2 as allowed failure

Until #429 is resolved, let us ignore openssl-1.0.2 failures
---
 .travis.yml | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index 9046e6dcd..a0d9502b4 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -99,6 +99,12 @@ matrix:
 before_script:
   - git clone http://git.1wt.eu/git/libslz.git/
   - cd libslz && make && make PREFIX=${HOME}/opt install && cd ..
+  allow_failures:
+  - os: linux
+arch: ppc64le
+if: type == cron
+compiler: gcc
+env: TARGET=linux-glibc OPENSSL_VERSION=1.0.2u
 
 install:
   - git clone https://github.com/VTest/VTest.git ../vtest
-- 
2.24.1

Re: [PATCH] introduce ARM64 travis-ci builds

[Илья Шипицин]

ср, 22 янв. 2020 г. в 10:49, Willy Tarreau :

> On Sun, Jan 19, 2020 at 12:18:00PM +0500,  ??? wrote:
> > hello,
> >
> > sometimes arm64 builds fails, I think it is good chance to introduce
> > regular builds
> > and fix them.
> >
> > also, few small improvements.
>


occasionely, I introduced "socat" for linux builds. god knows, I only
wanted socat on osx.

now we have daily failing openssl-1.0.2:

https://travis-ci.com/haproxy/haproxy/jobs/278244081

should I temporarily mark it as "allowed failure" on travis ? (it will be
fixed in issue #429)




>
> Merged, thanks Ilya. Next time, please be stricter and split your
> additions and your small improvements as it can help track down issues
> or do partial reverts.
>
> Willy
>

Re: [PATCH] improving ssl defines

[Илья Шипицин]

ср, 22 янв. 2020 г. в 11:24, Willy Tarreau :

> Hi Ilya,
>
> On Sat, Jan 18, 2020 at 06:47:48PM +0500,  ??? wrote:
> > Hello,
> >
> > let us switch to constants check, not versions.
> >
> > cheers,
> > Ilya Shipitcin
>
> > From a8b68e746bb71c4fee65a05bea8287ad970c979c Mon Sep 17 00:00:00 2001
> > From: Ilya Shipitsin 
> > Date: Sat, 18 Jan 2020 18:42:45 +0500
> > Subject: [PATCH] BUG/MINOR: ssl: fix build on openssl-1.1.0
> >
> > while working on issue #429, I built various openssl versions,
> > let us improve ssl defines, switch to features, not versions
>
> What is the build issue you've encountered exactly (and/or in what
> specific environment), and should this build fix be backported, and if
> so, how far ?
>
>
I think it should not be backported.

I've taken openssl branches (master, openssl_1_1_1 and openssl_1_1_0) and
built haproxy against various commits (not tagged as releases).
sometimes build fail, I ended with changing "ifdef" from
version-dependendent (which is not accurate) to feature dependent.

I can find particular commits, but I think it is not important since
released versions of openssl build just fine (I'll provide detailed info if
build will ever fail on openssl release).



> Thanks,
> Willy
>

Re: [PATCH] BUG/MINOR: dns: Make dns_query_id_seed unsigned

[Илья Шипицин]

вс, 19 янв. 2020 г. в 14:09, Willy Tarreau :

> On Sun, Jan 19, 2020 at 12:38:43PM +0500,  ??? wrote:
> > > > how noisy is ubsan ? should we add it to CI ?
> > > >
> > >
> > > No idea. I created a dedicated test executable and copied over the
> > > implementation to avoid needing to configure HAProxy to use DNS for the
> > > verification of my patch.
> > >
> > > In short: Simply try for yourself, please :-)
> > >
> >
> > it's bloody murder
> >
> > https://travis-ci.com/chipitsine/haproxy/jobs/277286655
>
> Thanks for the test. This report is not much encouraging considering
> that it shows the test confuses pointer calculations and pointer
> dereferences. Indeed, I've checked them all and they're all the
> byproduct of calling LIST_ELEM to retrieve a list element from the
> list's pointer, which is done by subtracting an offset from this
> pointer. There is absolutely no memory access done there. Thus I
> guess this rules out any hope to use this in automated tests :-)
>

ubsan actually includes several sanitizers. I'll try them one by one.


>
> Willy
>

Re: ARM(64) builds

[Илья Шипицин]

вс, 19 янв. 2020 г. в 13:13, Martin Grigorov :

> Hi,
>
> On Sat, Jan 18, 2020, 22:10 Илья Шипицин  wrote:
>
>> tests on ARM64 randomly fail
>> https://travis-ci.com/chipitsine/haproxy/jobs/277236120
>>
>> (after restart there's a good chance to success)
>>
>
> I have the same observation on TravisCI. I think the reason is that their
> arm64 instances are less powerful than the amd64 ones.
> At https://docs.travis-ci.com/user/multi-cpu-architectures it is said:
> While available to all Open Source repositories, the concurrency available
> for multiple CPU arch-based jobs is limited during the alpha period.
> Not sure how much limited it is though.
>

I'd like to investigate that on dedicated ARM64 server (which I currently
does not have).
I applied for one at Linode, no answer yet.


also, it might depend on number of CPU (race condition likely reproduce on
many CPU than
on single CPU server).

also, arm64 seems not very happy about our linker options

/usr/bin/ld: src/ev_poll.o(.debug_info+0x78): R_AARCH64_ABS64 used
with TLS symbol poll_events



>
> Today this test failed once on my VM:
>
> #top  TEST reg-tests/seamless-reload/abns_socket.vtc FAILED (1.111)
> exit=2
> 1 tests failed, 0 tests skipped, 34 tests passed
> ## Gathering results ##
> ## Test case: reg-tests/seamless-reload/abns_socket.vtc ##
> ## test results in:
> "/tmp/haregtests-2020-01-19_08-06-39.45Hchw/vtc.8496.328d7f95"
>  c1   HTTP rx failed (fd:6 read: Connection reset by peer)
> Makefile:964: recipe for target 'reg-tests' failed
> make: *** [reg-tests] Error 1
>
> but the next 4 runs were successul.
>
>
>> сб, 18 янв. 2020 г. в 09:52, Martin Grigorov :
>>
>>>
>>>
>>> On Fri, Jan 17, 2020 at 11:17 PM Martin Grigorov <
>>> martin.grigo...@gmail.com> wrote:
>>>
>>>>
>>>>
>>>> On Fri, Jan 17, 2020, 23:12 William Lallemand 
>>>> wrote:
>>>>
>>>>> On Fri, Jan 17, 2020 at 08:50:27PM +0200, Martin Grigorov wrote:
>>>>> > Testing with haproxy version: 2.2-dev0-70c5b0-123
>>>>>
>>>>> This binary was built with code from 1 week ago, it's normal that the
>>>>> test does
>>>>> not work since the fix was made this week.
>>>>>
>>>>
>>>> I'm using the same steps to build HAProxy as from .travis-ci.yml
>>>> I guess I have to add "make clean" in the beginning.
>>>> I'll try it tomorrow! Thanks!
>>>>
>>>
>>> That was it!
>>> Everything is back to normal now!
>>> Thank you, William!
>>>
>>>
>>>>
>>>>
>>>>> --
>>>>> William Lallemand
>>>>>
>>>>

Re: [PATCH] BUG/MINOR: dns: Make dns_query_id_seed unsigned

[Илья Шипицин]

сб, 18 янв. 2020 г. в 15:30, Tim Düsterhus :

> Ilya,
>
> Am 18.01.20 um 10:02 schrieb Илья Шипицин:
> > how noisy is ubsan ? should we add it to CI ?
> >
>
> No idea. I created a dedicated test executable and copied over the
> implementation to avoid needing to configure HAProxy to use DNS for the
> verification of my patch.
>
> In short: Simply try for yourself, please :-)
>

it's bloody murder

https://travis-ci.com/chipitsine/haproxy/jobs/277286655


>
> Best regards
> Tim Düsterhus
>

[PATCH] better anti replay check

[Илья Шипицин]

Hello,

let us check constants, not openssl versions.

Cheers,
Ilya Shipitcin
From eab262bda04f0f0caf8020a6837a75cdd5821e94 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Sun, 19 Jan 2020 12:20:14 +0500
Subject: [PATCH] BUILD: ssl: more elegant anti-reply feature presence check

---
 src/ssl_sock.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/ssl_sock.c b/src/ssl_sock.c
index 95dbe4c89..8484040f5 100644
--- a/src/ssl_sock.c
+++ b/src/ssl_sock.c
@@ -4682,7 +4682,7 @@ ssl_sock_initial_ctx(struct bind_conf *bind_conf)
 #ifdef OPENSSL_IS_BORINGSSL
 	SSL_CTX_set_select_certificate_cb(ctx, ssl_sock_switchctx_cbk);
 	SSL_CTX_set_tlsext_servername_callback(ctx, ssl_sock_switchctx_err_cbk);
-#elif (HA_OPENSSL_VERSION_NUMBER >= 0x10101000L)
+#elif defined(SSL_OP_NO_ANTI_REPLAY)
 	if (bind_conf->ssl_conf.early_data)
 		SSL_CTX_set_options(ctx, SSL_OP_NO_ANTI_REPLAY);
 	SSL_CTX_set_client_hello_cb(ctx, ssl_sock_switchctx_cbk, NULL);
-- 
2.24.1

[PATCH] introduce ARM64 travis-ci builds

[Илья Шипицин]

hello,

sometimes arm64 builds fails, I think it is good chance to introduce
regular builds
and fix them.

also, few small improvements.

cheers,
Ilya Shipicin
From e13a8d0e0e324c49f6a34d74b889a1c3fae6b6d9 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Sun, 19 Jan 2020 12:14:02 +0500
Subject: [PATCH] BUILD: CI: introduce ARM64 builds

also several small changes:

openssl-1.0.2 upgraded to 1.0.2u
ppc64le upgraded to "bionic" (it was tricky part, linux-ppc64le is xenial,
while arch: ppc64le is bionic).
additional wait introduced for build ssl.
---
 .travis.yml | 19 +++
 1 file changed, 15 insertions(+), 4 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index e69f44166..9046e6dcd 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -20,7 +20,11 @@ env:
 
 addons:
   apt:
-packages: [ liblua5.3-dev, libsystemd-dev, libpcre2-dev ]
+update: true
+packages: [ liblua5.3-dev, libsystemd-dev, libpcre2-dev, socat ]
+  homebrew:
+update: true
+packages: [ socat ]
 
 cache:
   directories:
@@ -29,11 +33,18 @@ cache:
 
 matrix:
   include:
-  - os: linux-ppc64le
+  - os: linux
+arch: ppc64le
 if: type == cron
 compiler: gcc
-env: TARGET=linux-glibc OPENSSL_VERSION=1.0.2t LABEL="linux-ppc64le"
+env: TARGET=linux-glibc OPENSSL_VERSION=1.0.2u
+  - os: linux
+arch: amd64
+if: type != cron
+compiler: clang
+env: TARGET=linux-glibc OPENSSL_VERSION=1.1.1d
   - os: linux
+arch: arm64
 if: type != cron
 compiler: clang
 env: TARGET=linux-glibc OPENSSL_VERSION=1.1.1d
@@ -93,7 +104,7 @@ install:
   - git clone https://github.com/VTest/VTest.git ../vtest
   # Special flags due to: https://github.com/vtest/VTest/issues/12
   - make -C ../vtest FLAGS="-O2 -s -Wall"
-  - scripts/build-ssl.sh > build-ssl.log 2>&1 || (cat build-ssl.log && exit 1)
+  - travis_wait scripts/build-ssl.sh > build-ssl.log 2>&1 || (cat build-ssl.log && exit 1)
 
 script:
   - if [ "$CC"  = "clang" ]; then export FLAGS="$FLAGS USE_OBSOLETE_LINKER=1" DEBUG_CFLAGS="-g -fsanitize=address" LDFLAGS="-fsanitize=address"; fi
-- 
2.24.1

Re: ARM(64) builds

[Илья Шипицин]

tests on ARM64 randomly fail
https://travis-ci.com/chipitsine/haproxy/jobs/277236120

(after restart there's a good chance to success)

сб, 18 янв. 2020 г. в 09:52, Martin Grigorov :

>
>
> On Fri, Jan 17, 2020 at 11:17 PM Martin Grigorov <
> martin.grigo...@gmail.com> wrote:
>
>>
>>
>> On Fri, Jan 17, 2020, 23:12 William Lallemand 
>> wrote:
>>
>>> On Fri, Jan 17, 2020 at 08:50:27PM +0200, Martin Grigorov wrote:
>>> > Testing with haproxy version: 2.2-dev0-70c5b0-123
>>>
>>> This binary was built with code from 1 week ago, it's normal that the
>>> test does
>>> not work since the fix was made this week.
>>>
>>
>> I'm using the same steps to build HAProxy as from .travis-ci.yml
>> I guess I have to add "make clean" in the beginning.
>> I'll try it tomorrow! Thanks!
>>
>
> That was it!
> Everything is back to normal now!
> Thank you, William!
>
>
>>
>>
>>> --
>>> William Lallemand
>>>
>>

[PATCH] improving ssl defines

[Илья Шипицин]

Hello,

let us switch to constants check, not versions.

cheers,
Ilya Shipitcin
From a8b68e746bb71c4fee65a05bea8287ad970c979c Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Sat, 18 Jan 2020 18:42:45 +0500
Subject: [PATCH] BUG/MINOR: ssl: fix build on openssl-1.1.0

while working on issue #429, I built various openssl versions,
let us improve ssl defines, switch to features, not versions
---
 include/common/openssl-compat.h | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/include/common/openssl-compat.h b/include/common/openssl-compat.h
index 72b4e2fe2..cb9caa3e9 100644
--- a/include/common/openssl-compat.h
+++ b/include/common/openssl-compat.h
@@ -284,8 +284,11 @@ static inline void EVP_PKEY_up_ref(EVP_PKEY *pkey)
 #define X509_getm_notAfter  X509_get_notAfter
 #endif
 
-#if (OPENSSL_VERSION_NUMBER < 0x101fL || defined LIBRESSL_VERSION_NUMBER)
+#if !defined(EVP_CTRL_AEAD_SET_IVLEN)
 #define EVP_CTRL_AEAD_SET_IVLEN EVP_CTRL_GCM_SET_IVLEN
+#endif
+
+#if !defined(EVP_CTRL_AEAD_SET_TAG)
 #define EVP_CTRL_AEAD_SET_TAG   EVP_CTRL_GCM_SET_TAG
 #endif
 
-- 
2.24.1

Re: [PATCH] BUG/MINOR: dns: Make dns_query_id_seed unsigned

[Илья Шипицин]

how noisy is ubsan ? should we add it to CI ?

сб, 18 янв. 2020 г. в 06:07, Tim Duesterhus :

> Left shifting of large signed values and negative values is undefined.
>
> In a test script clang's ubsan rightfully complains:
>
> > runtime error: left shift of 1934242336581872173 by 13 places cannot be
> represented in type 'int64_t' (aka 'long')
>
> This bug was introduced in the initial version of the DNS resolver
> in 325137d603aa81bd24cbd8c99d816dd42291daa7. The fix must be backported
> to HAProxy 1.6+.
> ---
>  src/dns.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/src/dns.c b/src/dns.c
> index bc68a81c0..64bb0d15f 100644
> --- a/src/dns.c
> +++ b/src/dns.c
> @@ -54,7 +54,7 @@
>  struct list dns_resolvers  = LIST_HEAD_INIT(dns_resolvers);
>  struct list dns_srvrq_list = LIST_HEAD_INIT(dns_srvrq_list);
>
> -static THREAD_LOCAL int64_t dns_query_id_seed = 0; /* random seed */
> +static THREAD_LOCAL uint64_t dns_query_id_seed = 0; /* random seed */
>
>  DECLARE_STATIC_POOL(dns_answer_item_pool, "dns_answer_item",
> sizeof(struct dns_answer_item));
>  DECLARE_STATIC_POOL(dns_resolution_pool,  "dns_resolution",
> sizeof(struct dns_resolution));
> --
> 2.25.0
>
>
>

Re: ARM(64) builds

[Илья Шипицин]

пт, 17 янв. 2020 г. в 19:33, Martin Grigorov :

>
>
> On Fri, Jan 17, 2020 at 4:13 PM Martin Grigorov 
> wrote:
>
>> Hi all,
>>
>> Today's build consistently fails on my ARM64 VM:
>>
>> ## Starting vtest ##
>> Testing with haproxy version: 2.2-dev0-70c5b0-123
>> #top  TEST reg-tests/mcli/mcli_start_progs.vtc FAILED (3.004) exit=2
>> 1 tests failed, 0 tests skipped, 34 tests passed
>> ## Gathering results ##
>> ## Test case: reg-tests/mcli/mcli_start_progs.vtc ##
>> ## test results in:
>> "/tmp/haregtests-2020-01-17_14-01-45.SGkYcJ/vtc.12807.6adfff44"
>>  h1   haproxy h1 PID file check failed:
>>  h1   Bad exit status: 0x0100 exit 0x1 signal 0 core 0
>> Makefile:964: recipe for target 'reg-tests' failed
>> make: *** [reg-tests] Error 1
>>
>
> git bisect blaims this commit:
>
> 25b569302167e71b32e569a2366027e8e320e80a is the first bad commit
> commit 25b569302167e71b32e569a2366027e8e320e80a
> Author: William Lallemand 
> Date:   Tue Jan 14 15:38:43 2020 +0100
>
> REGTEST: mcli/mcli_start_progs: start 2 programs
>
> This regtest tests the issue #446 by starting 2 programs and checking
> if
> they exist in the "show proc" of the master CLI.
>
> Should be backported as far as 2.0.
>
>
> https://travis-ci.com/haproxy/haproxy is green
> https://cirrus-ci.com/github/haproxy/haproxy is green
> and what is even more interesting is that
> https://travis-ci.org/martin-g/haproxy/builds (my fork with enabled ARM64
> testing on TravisCI) also just passed (after few failures due to timing
> issues (I guess)
>


timing out might happen because of
https://github.com/haproxy/haproxy/commit/ac8147446c7a3d1aa607042bc782095b03bc8dc4
your fork is 16 commits behind current master. try to rebase to master


>
>
>> Regards,
>> Martin
>>
>> On Fri, Jan 17, 2020 at 9:22 AM Илья Шипицин 
>> wrote:
>>
>>> привет!
>>>
>>> пт, 17 янв. 2020 г. в 11:42, Martin Grigorov >> >:
>>>
>>>> Привет Илья,
>>>>
>>>> On Thu, Jan 16, 2020 at 10:37 AM Илья Шипицин 
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> чт, 16 янв. 2020 г. в 13:26, Martin Grigorov <
>>>>> martin.grigo...@gmail.com>:
>>>>>
>>>>>> Hi Илья,
>>>>>>
>>>>>> On Thu, Jan 16, 2020 at 10:19 AM Илья Шипицин 
>>>>>> wrote:
>>>>>>
>>>>>>>
>>>>>>> Hello, Martin!
>>>>>>>
>>>>>>> btw, just curious, how is Apache Foundation (or you personally)
>>>>>>> interested in all that ?
>>>>>>> please do not blame me, I really like to know.
>>>>>>>
>>>>>>
>>>>> ok, so you work in some company that is interested in haproxy on
>>>>> ARM64.
>>>>> you do not want to tell it's name, at least is it legal ? is it
>>>>> related to some government ?
>>>>> if "no" and "no", I guess most people won't ask any more questions :)
>>>>>
>>>>
>>>> It is legal and I do not work for a government of any country!
>>>>
>>>>
>>>>>
>>>>> personally, I do not work at Haproxy Inc, I use haproxy, sometimes I
>>>>> contribute to it.
>>>>> Please do not consider me as an "official representative".
>>>>>
>>>>>
>>>>> I'm interested in testing haproxy on ARM64, I planned to do so. I can
>>>>> priorierize it according to your interest to it.
>>>>> and yes, I can accept hardware donation (even considering I'm not part
>>>>> of Haproxy Inc).
>>>>>
>>>>> also, from my point of view, what would be really useful in your case
>>>>> is testing not just official reg-tests, but also
>>>>> your configs. reg-tests cover only partially. If you enable clang asan
>>>>> in your own workload there's a chance to catch
>>>>> something interesting (or, to make sure your own workload is ok). I
>>>>> can help with that as well.
>>>>>
>>>>
>>>> Thanks for the offer!
>>>> I've discussed it with my managers.
>>>> Our offer is to donate a VM that could be used as an official CI agen

Re: ARM(64) builds

[Илья Шипицин]

привет!

пт, 17 янв. 2020 г. в 11:42, Martin Grigorov :

> Привет Илья,
>
> On Thu, Jan 16, 2020 at 10:37 AM Илья Шипицин 
> wrote:
>
>>
>>
>> чт, 16 янв. 2020 г. в 13:26, Martin Grigorov :
>>
>>> Hi Илья,
>>>
>>> On Thu, Jan 16, 2020 at 10:19 AM Илья Шипицин 
>>> wrote:
>>>
>>>>
>>>> Hello, Martin!
>>>>
>>>> btw, just curious, how is Apache Foundation (or you personally)
>>>> interested in all that ?
>>>> please do not blame me, I really like to know.
>>>>
>>>
>> ok, so you work in some company that is interested in haproxy on ARM64.
>> you do not want to tell it's name, at least is it legal ? is it related
>> to some government ?
>> if "no" and "no", I guess most people won't ask any more questions :)
>>
>
> It is legal and I do not work for a government of any country!
>
>
>>
>> personally, I do not work at Haproxy Inc, I use haproxy, sometimes I
>> contribute to it.
>> Please do not consider me as an "official representative".
>>
>>
>> I'm interested in testing haproxy on ARM64, I planned to do so. I can
>> priorierize it according to your interest to it.
>> and yes, I can accept hardware donation (even considering I'm not part of
>> Haproxy Inc).
>>
>> also, from my point of view, what would be really useful in your case is
>> testing not just official reg-tests, but also
>> your configs. reg-tests cover only partially. If you enable clang asan in
>> your own workload there's a chance to catch
>> something interesting (or, to make sure your own workload is ok). I can
>> help with that as well.
>>
>
> Thanks for the offer!
> I've discussed it with my managers.
> Our offer is to donate a VM that could be used as an official CI agent for
> the HAProxy project long term.
>

I'd split this into short term and long term approach.

if you need to start any time soon, I'd focus on your own workload testing
first. I would build stand which emulates
your production workload, compile haproxy using clang address sanitizer and
give it a try (functional testing, load testing, ...)

I can help with that.

As for long term solution, currently haproxy simply cannot attach any
dedicated build agent to their CI. travis-ci does not allow
attaching dedicated agents. And haproxy team is very conservative in adding
new CI servers.

I think, I will add arm64 (most probably openssl-1.1.1 only for now) soon.
Also, I'm going to investigate your libressl failures.

so, dedicated vm definetly will help in troubleshooting issues, for manual
builds. It would save bunch of time. I do not mind if you will
add my ssh to that vm.


also, I requested access to Linaro.


>
> You can use Linaro for short term testing though.
> https://www.linaro.cloud/
> Here you can request free VM for short periods:
> https://servicedesk.linaro.org/servicedesk/customer/portal/19/create/265
> P.S. Linaro is not my employer!
>
>
> Regards,
> Martin
>
>
>>
>>>
>>>>
>>> @apache.org is just one of my several emails. And it is the default one
>>> in my email client.
>>> ASF is not related anyhow to my participation here.
>>> If I used my work email then it might look like some kind of
>>> advertisement. I'd like to avoid that!
>>> Next time I will use my @gmail.com one, as more neutral. Actually I've
>>> used the GMail one when registering to this mailing list, so probably the
>>> post from @apache has been moderated. I'll be more careful next time!
>>>
>>> Thanks, Илья!
>>>
>>> Regards,
>>> Martin
>>>
>>>
>>>>
>>>> чт, 16 янв. 2020 г. в 12:32, Martin Grigorov :
>>>>
>>>>> Hello HAProxy developers,
>>>>>
>>>>> At work we are going to use more and more ARM64 based servers and
>>>>> HAProxy is one of the main products we rely on.
>>>>> So I went checking whether HAProxy project has a CI environment for
>>>>> testing on ARM architecture.
>>>>>
>>>>
>>>> we are looking towards
>>>> https://docs.travis-ci.com/user/multi-cpu-architectures
>>>>
>>>>
>>>>> I've found this recent discussion:
>>>>> https://www.mail-archive.com/haproxy@formilux.org/msg35302.html  (I
>>>>> didn't find a way how to continue on the same mail thread, so I'm starting
>>>>> a new one. Apologies for that!).
>>>>>
>>>>
>>>> I played with

Re: ARM(64) builds

[Илья Шипицин]

чт, 16 янв. 2020 г. в 13:37, Илья Шипицин :

>
>
> чт, 16 янв. 2020 г. в 13:26, Martin Grigorov :
>
>> Hi Илья,
>>
>> On Thu, Jan 16, 2020 at 10:19 AM Илья Шипицин 
>> wrote:
>>
>>>
>>> Hello, Martin!
>>>
>>> btw, just curious, how is Apache Foundation (or you personally)
>>> interested in all that ?
>>> please do not blame me, I really like to know.
>>>
>>
> ok, so you work in some company that is interested in haproxy on ARM64.
> you do not want to tell it's name, at least is it legal ? is it related to
> some government ?
> if "no" and "no", I guess most people won't ask any more questions :)
>

oops. I meant "yes" and "no"


>
> personally, I do not work at Haproxy Inc, I use haproxy, sometimes I
> contribute to it.
> Please do not consider me as an "official representative".
>
>
> I'm interested in testing haproxy on ARM64, I planned to do so. I can
> priorierize it according to your interest to it.
> and yes, I can accept hardware donation (even considering I'm not part of
> Haproxy Inc).
>
> also, from my point of view, what would be really useful in your case is
> testing not just official reg-tests, but also
> your configs. reg-tests cover only partially. If you enable clang asan in
> your own workload there's a chance to catch
> something interesting (or, to make sure your own workload is ok). I can
> help with that as well.
>
>
>>
>>>
>> @apache.org is just one of my several emails. And it is the default one
>> in my email client.
>> ASF is not related anyhow to my participation here.
>> If I used my work email then it might look like some kind of
>> advertisement. I'd like to avoid that!
>> Next time I will use my @gmail.com one, as more neutral. Actually I've
>> used the GMail one when registering to this mailing list, so probably the
>> post from @apache has been moderated. I'll be more careful next time!
>>
>> Thanks, Илья!
>>
>> Regards,
>> Martin
>>
>>
>>>
>>> чт, 16 янв. 2020 г. в 12:32, Martin Grigorov :
>>>
>>>> Hello HAProxy developers,
>>>>
>>>> At work we are going to use more and more ARM64 based servers and
>>>> HAProxy is one of the main products we rely on.
>>>> So I went checking whether HAProxy project has a CI environment for
>>>> testing on ARM architecture.
>>>>
>>>
>>> we are looking towards
>>> https://docs.travis-ci.com/user/multi-cpu-architectures
>>>
>>>
>>>> I've found this recent discussion:
>>>> https://www.mail-archive.com/haproxy@formilux.org/msg35302.html  (I
>>>> didn't find a way how to continue on the same mail thread, so I'm starting
>>>> a new one. Apologies for that!).
>>>>
>>>
>>> I played with arm64 for a while, the issue I encountered is travis-ci
>>> cache key, i.e. we cache openssl builds between our builds.
>>> so travis used the same cache key for both amd64 and arm64 builds (this
>>> might have changed recently, I did not check yet)
>>>
>>> arm64 is in my queue (as well as recent s390x arch from travis), hope to
>>> get back to it within month or so.
>>>
>>>
>>>> From this discussion and from
>>>> https://github.com/haproxy/haproxy/blob/master/.travis.yml I
>>>> understand that there is no public CI in use (i.e. TravisCI or CirrusCI)
>>>> but some of the developers run some tests locally regularly.
>>>>
>>>
>>> it is not completely true.
>>> there's public CI. we do not use github PR machinery, so sometimes tests
>>> fail after push to master branch. it is considered as ok, failures are
>>> fixed pretty fast.
>>> for example, see
>>> https://www.mail-archive.com/haproxy@formilux.org/msg35910.html
>>> it was just perfect, failure detected using CI and fixed within few
>>> days. no customers affected.
>>>
>>>
>>>> I've forked the project and tested on TravisCI (
>>>> https://travis-ci.org/martin-g/haproxy/builds) but unfortunately the
>>>> builds were not very stable:
>>>> 1) some tests fail sometimes. I guess it is because of some timing
>>>> issues
>>>> For example:
>>>> - https://travis-ci.org/martin-g/haproxy/jobs/636745241
>>>> - https://travis-ci.org/martin-g/haproxy/jobs/636750676
>>>> - https://travis-ci.org/martin-g/haproxy/jobs/636763346
>>

Re: ARM(64) builds

[Илья Шипицин]

чт, 16 янв. 2020 г. в 13:26, Martin Grigorov :

> Hi Илья,
>
> On Thu, Jan 16, 2020 at 10:19 AM Илья Шипицин 
> wrote:
>
>>
>> Hello, Martin!
>>
>> btw, just curious, how is Apache Foundation (or you personally)
>> interested in all that ?
>> please do not blame me, I really like to know.
>>
>
ok, so you work in some company that is interested in haproxy on ARM64.
you do not want to tell it's name, at least is it legal ? is it related to
some government ?
if "no" and "no", I guess most people won't ask any more questions :)

personally, I do not work at Haproxy Inc, I use haproxy, sometimes I
contribute to it.
Please do not consider me as an "official representative".


I'm interested in testing haproxy on ARM64, I planned to do so. I can
priorierize it according to your interest to it.
and yes, I can accept hardware donation (even considering I'm not part of
Haproxy Inc).

also, from my point of view, what would be really useful in your case is
testing not just official reg-tests, but also
your configs. reg-tests cover only partially. If you enable clang asan in
your own workload there's a chance to catch
something interesting (or, to make sure your own workload is ok). I can
help with that as well.


>
>>
> @apache.org is just one of my several emails. And it is the default one
> in my email client.
> ASF is not related anyhow to my participation here.
> If I used my work email then it might look like some kind of
> advertisement. I'd like to avoid that!
> Next time I will use my @gmail.com one, as more neutral. Actually I've
> used the GMail one when registering to this mailing list, so probably the
> post from @apache has been moderated. I'll be more careful next time!
>
> Thanks, Илья!
>
> Regards,
> Martin
>
>
>>
>> чт, 16 янв. 2020 г. в 12:32, Martin Grigorov :
>>
>>> Hello HAProxy developers,
>>>
>>> At work we are going to use more and more ARM64 based servers and
>>> HAProxy is one of the main products we rely on.
>>> So I went checking whether HAProxy project has a CI environment for
>>> testing on ARM architecture.
>>>
>>
>> we are looking towards
>> https://docs.travis-ci.com/user/multi-cpu-architectures
>>
>>
>>> I've found this recent discussion:
>>> https://www.mail-archive.com/haproxy@formilux.org/msg35302.html  (I
>>> didn't find a way how to continue on the same mail thread, so I'm starting
>>> a new one. Apologies for that!).
>>>
>>
>> I played with arm64 for a while, the issue I encountered is travis-ci
>> cache key, i.e. we cache openssl builds between our builds.
>> so travis used the same cache key for both amd64 and arm64 builds (this
>> might have changed recently, I did not check yet)
>>
>> arm64 is in my queue (as well as recent s390x arch from travis), hope to
>> get back to it within month or so.
>>
>>
>>> From this discussion and from
>>> https://github.com/haproxy/haproxy/blob/master/.travis.yml I understand
>>> that there is no public CI in use (i.e. TravisCI or CirrusCI) but some of
>>> the developers run some tests locally regularly.
>>>
>>
>> it is not completely true.
>> there's public CI. we do not use github PR machinery, so sometimes tests
>> fail after push to master branch. it is considered as ok, failures are
>> fixed pretty fast.
>> for example, see
>> https://www.mail-archive.com/haproxy@formilux.org/msg35910.html
>> it was just perfect, failure detected using CI and fixed within few days.
>> no customers affected.
>>
>>
>>> I've forked the project and tested on TravisCI (
>>> https://travis-ci.org/martin-g/haproxy/builds) but unfortunately the
>>> builds were not very stable:
>>> 1) some tests fail sometimes. I guess it is because of some timing issues
>>> For example:
>>> - https://travis-ci.org/martin-g/haproxy/jobs/636745241
>>> - https://travis-ci.org/martin-g/haproxy/jobs/636750676
>>> - https://travis-ci.org/martin-g/haproxy/jobs/636763346
>>>
>>
>> that's very interesting. I'll have a look.
>>
>>
>>
>>> 2) There was some weird issue on testing with LibreSSL
>>> The output redirect at
>>> https://github.com/haproxy/haproxy/blob/bb9da0b8e23c46caab34fe6005b66fa8065fe3ac/.travis.yml#L96
>>>  for
>>> some reason got stuck the build. I've removed temporarily the output
>>> redirects and then it passed. So, it looks like some issue with TravisCI
>>> environment.
>>>
>>
>> arm64 is slower, I gue

Re: ARM(64) builds

[Илья Шипицин]

Hello, Martin!

btw, just curious, how is Apache Foundation (or you personally) interested
in all that ?
please do not blame me, I really like to know.


чт, 16 янв. 2020 г. в 12:32, Martin Grigorov :

> Hello HAProxy developers,
>
> At work we are going to use more and more ARM64 based servers and HAProxy
> is one of the main products we rely on.
> So I went checking whether HAProxy project has a CI environment for
> testing on ARM architecture.
>

we are looking towards
https://docs.travis-ci.com/user/multi-cpu-architectures


> I've found this recent discussion:
> https://www.mail-archive.com/haproxy@formilux.org/msg35302.html  (I
> didn't find a way how to continue on the same mail thread, so I'm starting
> a new one. Apologies for that!).
>

I played with arm64 for a while, the issue I encountered is travis-ci cache
key, i.e. we cache openssl builds between our builds.
so travis used the same cache key for both amd64 and arm64 builds (this
might have changed recently, I did not check yet)

arm64 is in my queue (as well as recent s390x arch from travis), hope to
get back to it within month or so.


> From this discussion and from
> https://github.com/haproxy/haproxy/blob/master/.travis.yml I understand
> that there is no public CI in use (i.e. TravisCI or CirrusCI) but some of
> the developers run some tests locally regularly.
>

it is not completely true.
there's public CI. we do not use github PR machinery, so sometimes tests
fail after push to master branch. it is considered as ok, failures are
fixed pretty fast.
for example, see
https://www.mail-archive.com/haproxy@formilux.org/msg35910.html
it was just perfect, failure detected using CI and fixed within few days.
no customers affected.


> I've forked the project and tested on TravisCI (
> https://travis-ci.org/martin-g/haproxy/builds) but unfortunately the
> builds were not very stable:
> 1) some tests fail sometimes. I guess it is because of some timing issues
> For example:
> - https://travis-ci.org/martin-g/haproxy/jobs/636745241
> - https://travis-ci.org/martin-g/haproxy/jobs/636750676
> - https://travis-ci.org/martin-g/haproxy/jobs/636763346
>

that's very interesting. I'll have a look.



> 2) There was some weird issue on testing with LibreSSL
> The output redirect at
> https://github.com/haproxy/haproxy/blob/bb9da0b8e23c46caab34fe6005b66fa8065fe3ac/.travis.yml#L96
>  for
> some reason got stuck the build. I've removed temporarily the output
> redirects and then it passed. So, it looks like some issue with TravisCI
> environment.
>

arm64 is slower, I guess we should add "*travis*_*wait* *30" *to
build-ssl.sh script
thank for the hint


>
> In addition I've run the build and tests on one of our machines and all
> was OK!
>
> My question to you is: Are you happy with your current way of testing ARM
> architectures or you want to add more ?
> Here are some options:
> 1) enable TravisCI
>

already done

https://travis-ci.com/haproxy/haproxy


> 2) my company is willing to donate an ARM64 based VM, if you are
> interested.
>

I do not work at Haproxy Inc :)
Willy ?


> You will have a SSH access and a user with sudo permissions to install
> anything that is needed.
> The spec is aarch64 8 cores CPU 2GHz (Kunpeng), 16GB RAM, 500G disk space
> and 5M network bandwidth. The OS could be any of CentOS 7.4/7.5/7.6,
> EulerOS 2.8, Fedora 29, OpenSuse 15.0 and Ubuntu 16.04/18.04.
>
> In both cases it will be ARM64. From the earlier mail discussion I
> understand you would prefer ARM32.
>

as for myself, I prefer both arm64 and arm32.
however, both AMD64 and ARM64 are the same 64 bits. both of them are
little-endian. but you mentioned at least 3 builds with ARM64 failing (we
have those tests passing on AMD64)!



>
> Kind regards,
> Martin
>

Re: freebsd ci is broken - commit 08fa16e - curl download stalls in reg-tests/compression/lua_validation.vtc

[Илья Шипицин]

ср, 15 янв. 2020 г. в 04:18, PiBa-NL :

> Hi Ilya, Willy,
>
> Op 14-1-2020 om 21:40 schreef Илья Шипицин:
> > PiBa, how many CPU cores are you running ?
> >
> > it turned out that I run tests on very low vm, which only has 1 core.
> > and tests pass.
> > cirrus-ci as far as I remember do have many cores.
> I was running with 16 cores..
> >
> > can you find single core vm ?
>
> Well, i reconfigured the VM to have 1 core, but same issue seems to show
> up, though not on every time the test is run, and actually a bit less
> often..
> Below some additional testresults with different kqueue / vCPU settings..
>

I run tests on MS Azure B1s vm, it is cheapest size. Single "shared" core.
Tests always pass.
looks like we've found race condition :)


>
>
> *VM with 1 vCPU*
>
> Running: ./vtest/VTest-master/vtest -Dno-htx=no -l -k -b 50M -t 5 -n 20
> ./work/haproxy-08fa16e/reg-tests/compression/lua_validation.vtc
>Results in: 4 tests failed, 0 tests skipped, 16 tests passed
>
> Adding "nokqueue" in the vtc file i get:
>8 tests failed, 0 tests skipped, 12 tests passed
>4 tests failed, 0 tests skipped, 16 tests passed
>
> So its a bit random, but the 'nokqueue' directive does not seem to
> affect results much..
>
>
> *With 16 vCPU*
> Without nokqueue: 16 tests failed, 0 tests skipped, 4 tests passed
> With nokqueue (using poll): 17 tests failed, 0 tests skipped, 3 tests
> passed
>
> The failure rate seems certainly higher with many cores..
>
>
> * Using commit 0eae632 it works OK*
> Just to be sure i re-tested on 16 cores with 2.2-dev0-0eae632 but that
> does nicely pass: 0 tests failed, 0 tests skipped, 20 tests passed
>
> Regards,
> PiBa-NL (Pieter)
>
>

Re: freebsd ci is broken - commit 08fa16e - curl download stalls in reg-tests/compression/lua_validation.vtc

[Илья Шипицин]

PiBa, how many CPU cores are you running ?

it turned out that I run tests on very low vm, which only has 1 core. and
tests pass.
cirrus-ci as far as I remember do have many cores.

can you find single core vm ?

ср, 15 янв. 2020 г. в 00:02, PiBa-NL :

> Hi Ilya,
>
> Thanks!
>
> Op 14-1-2020 om 07:48 schreef Илья Шипицин:
>
> Hello,
>
> since
>
> https://github.com/haproxy/haproxy/commit/08fa16e397ffb1c6511b98ade2a3bfff9435e521
>
> freebsd CI is red: https://cirrus-ci.com/task/5960933184897024
>
> I'd say "it is something with CI itself",  when I run the same tests
> locally on freebsd, it is green.
>
> Sadly i do get the same problem on my test server (version info below its
> version 11.1 is a bit outdated, but hasn't failed my before...).
>
>
> PiBa ?
>
>
> thanks,
> Ilya Shipitcin
>
> Below a part of the output that the test generates for me. The first curl
> request seems to succeed, but the second one runs into a timeout..
> When compiled with the commit before 08fa16e
> <https://github.com/haproxy/haproxy/commit/08fa16e397ffb1c6511b98ade2a3bfff9435e521>
> it does not show that behaviour.. Current latest(24c928c) commit is still
> affected..
>
>  top  shell_out|  % Total% Received % Xferd  Average Speed
> TimeTime Time  Current
>  top  shell_out| Dload  Upload
> Total   SpentLeft  Speed
>  top  shell_out|\r  0 00 00 0  0  0
> --:--:-- --:--:-- --:--:-- 0\r100  418k0  418k0 0
> 1908k  0 --:--:-- --:--:-- --:--:-- 1908k
>  top  shell_out|  % Total% Received % Xferd  Average Speed
> TimeTime Time  Current
>  top  shell_out| Dload  Upload
> Total   SpentLeft  Speed
>  top  shell_out|\r  0 00 00 0  0  0
> --:--:-- --:--:-- --:--:-- 0\r100  141k0  141k0 0
> 284k  0 --:--:-- --:--:-- --:--:--  284k\r100  343k0  343k0
> 0   156k  0 --:--:--  0:00:02 --:--:--  156k\r100  343k0  343k
> 0 0   105k  0 --:--:--  0:00:03 --:--:--  105k\r100  343k0
> 343k0 0  81274  0 --:--:--  0:00:04 --:--:-- 81274\r100
> 343k0  343k0 0  65228  0 --:--:--  0:00:05 --:--:--
> 65240\r100  343k0  343k0 0  54481  0 --:--:--  0:00:06
> --:--:-- 34743\r100  343k0  343k0 0  46768  0 --:--:--
> 0:00:07 --:--:-- 0\r100  343k0  343k0 0  40968  0
> --:--:--  0:00:08 --:--:-- 0\r100  343k0  343k0 0
> 36452  0 --:--:--  0:00:09 --:--:-- 0\r100  343k0  343k
> 0 0  32830  0 --:--:--  0:00:10 --:--:-- 0\r100  343k0
> 343k0 0  29865  0 --:--:--  0:00:11 --:--:-- 0\r100
> 343k0  343k0 0  27395  0 --:--:--  0:00:12 --:--:--
> 0\r100  343k0  343k0 0  25297  0 --:--:--  0:00:13
> --:--:-- 0\r100  343k0  343k0 0  23500  0 --:--:--
> 0:00:14 --:--:-- 0\r100  343k0  343k0 0  23431  0
> --:--:--  0:00:15 --:--:-- 0
>  top  shell_out|curl: (28) Operation timed out after 15002
> milliseconds with 351514 bytes received
>  top  shell_out|Expecting checksum 4d9c62aa5370b8d5f84f17ec2e78f483
>  top  shell_out|Received checksum: da2d120aedfd693eeba9cf1e578897a8
>  top  shell_status = 0x0001
>  top  shell_exit not as expected: got 0x0001 wanted 0x
> *top  RESETTING after
> ./work/haproxy-08fa16e/reg-tests/compression/lua_validation.vtc
>
> Should i update to a newer FreeBSD version, or is it likely unrelated, and
> in need of some developer attention.. Do you (Willy or anyone), need more
> information from my side? Or is there a patch i can try to validate?
>
> Regards,
> PiBa-NL (Pieter)
>
>
> Yes im running a somewhat outdated OS here:
>   FreeBSD freebsd11 11.1-RELEASE FreeBSD 11.1-RELEASE #0 r321309: Fri Jul
> 21 02:08:28 UTC 2017
> r...@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64
>
> Version used:
>   haproxy -vv
> HA-Proxy version 2.2-dev0-08fa16e 2020/01/08 - https://haproxy.org/
> Status: development branch - not safe for use in production.
> Known bugs: https://github.com/haproxy/haproxy/issues?q=is:issue+is:open
> Build options :
>   TARGET  = freebsd
>   CPU = generic
>   CC  = cc
>   CFLAGS  = -pipe -g -fstack-protector -fno-strict-aliasing
> -fno-strict-aliasing -Wdeclaration-after-statement -fwrapv
> -fno-strict-overflow -Wno-null-dereference -Wno-unused-label
> -Wno-unused-parameter -Wno-sign-compare -Wno-ignored-qualifiers
> -Wno-unused-command-line-argument -Wno-missing-field-initializers
> -Wno-address-of-packed-member -

freebsd ci is broken

[Илья Шипицин]

Hello,

since
https://github.com/haproxy/haproxy/commit/08fa16e397ffb1c6511b98ade2a3bfff9435e521

freebsd CI is red: https://cirrus-ci.com/task/5960933184897024

I'd say "it is something with CI itself",  when I run the same tests
locally on freebsd, it is green.

PiBa ?


thanks,
Ilya Shipitcin

Re: [PATCH] CLEANUP: server: remove unused err section in server_finalize_init

[Илья Шипицин]

btw, if you add "Fixes: #438", the issue will be closed automatically

https://help.github.com/en/github/managing-your-work-on-github/closing-issues-using-keywords

чт, 9 янв. 2020 г. в 01:33, William Dauchy :

> Since commit 980855bd953c ("BUG/MEDIUM: server: initialize the orphaned
> conns lists and tasks at the end"), we no longer use err section.
>
> This should fix github issue #438
>
> Signed-off-by: William Dauchy 
> ---
>  src/server.c | 2 --
>  1 file changed, 2 deletions(-)
>
> diff --git a/src/server.c b/src/server.c
> index 6212a420..14ff716a 100644
> --- a/src/server.c
> +++ b/src/server.c
> @@ -2053,8 +2053,6 @@ static int server_finalize_init(const char *file,
> int linenum, char **args, int
> srv_lb_commit_status(srv);
>
> return 0;
> -err:
> -   return ERR_ALERT | ERR_FATAL;
>  }
>
>  /*
> --
> 2.24.1
>
>
>

[PATCH] adapt regtest for travis-ci

[Илья Шипицин]

Hello,

for some reason "echo -e" does not work at least in travis-ci.
as discussed in #423 it is good to use "printf" instead.

Cheers,
Ilya Shipitcin
From e0ae24b754b6f660b0d942df25ed1e0bfe189967 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Wed, 8 Jan 2020 22:56:30 +0500
Subject: [PATCH] REGTEST: set_ssl_cert.vtc: replace "echo" with "printf"

"echo -e" for some reason does not work on travis-ci, so let us switch
to "printf"

Fixes: #423
---
 reg-tests/ssl/set_ssl_cert.vtc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/reg-tests/ssl/set_ssl_cert.vtc b/reg-tests/ssl/set_ssl_cert.vtc
index 3d6ff0c98..11b2d154b 100644
--- a/reg-tests/ssl/set_ssl_cert.vtc
+++ b/reg-tests/ssl/set_ssl_cert.vtc
@@ -42,7 +42,7 @@ shell {
 }
 
 shell {
-   echo -e "set ssl cert ${testdir}/common.pem <<\n$(cat ${testdir}/ecdsa.pem)\n" | socat "${tmpdir}/h1/stats" -
+   printf "set ssl cert ${testdir}/common.pem <<\n$(cat ${testdir}/ecdsa.pem)\n\n" | socat "${tmpdir}/h1/stats" -
echo "commit ssl cert ${testdir}/common.pem" | socat "${tmpdir}/h1/stats" -
 }
 
-- 
2.24.1

[PATCH] cirrus-ci: choose proper openssl package name

[Илья Шипицин]

hello,


freebsd-11.3 names package "openssl"
freebsd-12.1 names package "openssl111"

I added switch for that

Cheers,
Ilya Shipitcin
From 614d7497641f0a2459c5f0b584fc84345ba5aeee Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Tue, 7 Jan 2020 23:41:24 +0500
Subject: [PATCH] BUILD: cirrus-ci: choose proper openssl package name

freebsd-11.3 and 12.1 comes with different openssl naming
let us add proper switch to cirrus-ci script
---
 .cirrus.yml | 8 +++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index d14678111..4143e687b 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -5,7 +5,13 @@ FreeBSD_task:
   image_family: freebsd-11-3-snap
   only_if: $CIRRUS_BRANCH =~ 'master|next'
   install_script:
-- pkg install -y openssl111 git gmake lua53 socat
+- case `uname -r` in
+11.3*)
+  export SSL=openssl;;
+12.1*)
+  export SSL=openssl111;;
+  esac
+- pkg install -y $SSL git gmake lua53 socat
   script:
 - git clone https://github.com/VTest/VTest.git ../vtest
 - make -C ../vtest
-- 
2.24.1

learning SSL stuff

[Илья Шипицин]

Hello,

while working on #429,
I noticed strange things that I do not understand
for example

if (ecdhe == NULL) {
(void)SSL_CTX_set_ecdh_auto(ctx, 1);
return cfgerr;
}

why do we need to call SSL_CTX_set_ecdh_auto right before returning error ?


Cheers,
Ilya Shipitsin

[PATCH] modernize cirrus-ci

[Илья Шипицин]

Hello,

small refinement of cirrus-ci magic.

Cheers,
Ilya Shipicin
From bc6989debcd3b2e4a45f18d2a58e65fa4ab2f555 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Thu, 26 Dec 2019 13:37:36 +0500
Subject: [PATCH] BUILD: CI: modernize cirrus-ci

use freebsd-12.1 instead of freebsd-12.0,
add freebsd-11.3 to build matrix,
install socat in order to run modern reg-tests
---
 .cirrus.yml | 11 ++-
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index e4278c2ce..c13e1d8d4 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -1,11 +1,12 @@
 FreeBSD_task:
   freebsd_instance:
-image: freebsd-12-0-release-amd64
+matrix:
+  image_family: freebsd-12-1
+  image_family: freebsd-11-3-snap
   only_if: $CIRRUS_BRANCH =~ 'master|next'
-  env:
-ASSUME_ALWAYS_YES: TRUE # required for unattended "pkg" invocation
+  install_script: 
+- pkg install -y openssl111 git gmake lua53 socat
   script:
-- pkg install openssl111 git gmake lua53
 - git clone https://github.com/VTest/VTest.git ../vtest
 - make -C ../vtest
 - gmake CC=clang V=1 TARGET=freebsd USE_ZLIB=1 USE_PCRE=1 USE_OPENSSL=1 USE_LUA=1 LUA_INC=/usr/local/include/lua53 LUA_LIB=/usr/local/lib LUA_LIB_NAME=lua-5.3
@@ -18,7 +19,7 @@ centos_6_task:
 image: centos:centos6
   only_if: $CIRRUS_BRANCH =~ 'master|next'
   script:
-- yum install -q -y gcc git openssl-devel pcre-devel epel-release
+- yum install -q -y gcc git openssl-devel pcre-devel epel-release socat
 - yum install -q -y python34
 - git clone https://github.com/VTest/VTest.git ../vtest
 # Special flags due to: https://github.com/vtest/VTest/issues/12
-- 
2.24.1

[PATCH] reenable ASAN for travis-ci builds

[Илья Шипицин]

Hello,

ASAN stands for address sanitizer, it is powerful tool for detecting memory
allocation issues (it already detected several of them). ASAN was
temporarily disabled.

Now (I guess after getting rid of LD_LIBRARY_PATH) let us enable it again.

Cheers,
Ilya Shipitcin
From f8f3eb6eabed36dcad72f2330a4f056c5548168f Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Sat, 21 Dec 2019 18:10:28 +0500
Subject: [PATCH] BUILD: travis-ci: reenable address sanitizer for clang builds

address sanitizer was temporarily disabled. after getting rid of
LD_LIBRARY_PATH manipulation it works again, so let us enable it
---
 .travis.yml | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/.travis.yml b/.travis.yml
index 6df39f52b..e69f44166 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -96,8 +96,9 @@ install:
   - scripts/build-ssl.sh > build-ssl.log 2>&1 || (cat build-ssl.log && exit 1)
 
 script:
+  - if [ "$CC"  = "clang" ]; then export FLAGS="$FLAGS USE_OBSOLETE_LINKER=1" DEBUG_CFLAGS="-g -fsanitize=address" LDFLAGS="-fsanitize=address"; fi
   - make -C contrib/wurfl
-  - make -j3 CC=$CC V=1 TARGET=$TARGET $FLAGS DEBUG_CFLAGS="$DEBUG_CFLAGS" LDFLAGS="-L$SSL_LIB -Wl,-rpath,$SSL_LIB" 51DEGREES_SRC="$FIFTYONEDEGREES_SRC" EXTRA_OBJS="$EXTRA_OBJS"
+  - make -j3 CC=$CC V=1 TARGET=$TARGET $FLAGS DEBUG_CFLAGS="$DEBUG_CFLAGS" LDFLAGS="$LDFLAGS -L$SSL_LIB -Wl,-rpath,$SSL_LIB" 51DEGREES_SRC="$FIFTYONEDEGREES_SRC" EXTRA_OBJS="$EXTRA_OBJS"
   - ./haproxy -vv
   - if [ "${TRAVIS_OS_NAME}" = "linux" ]; then ldd haproxy; fi
   - if [ "${TRAVIS_OS_NAME}" = "osx" ]; then otool -L haproxy; fi
-- 
2.24.1

Re: PATCH: partially fix build if OpenSSL is built with no-deprecated option

[Илья Шипицин]

сб, 21 дек. 2019 г. в 01:44, Rosen Penev :

> On Fri, Dec 20, 2019 at 10:54 AM Илья Шипицин 
> wrote:
> >
> >
> >
> > пт, 20 дек. 2019 г. в 22:39, Lukas Tribus :
> >>
> >> Hello Ilya,
> >>
> >>
> >>
> >> sorry about the delay ...
> >>
> >>
> >> On Wed, 27 Nov 2019 at 07:11, Илья Шипицин 
> wrote:
> >> >
> >> > -#if (HA_OPENSSL_VERSION_NUMBER >= 0x101fL)
> >> > +#if (HA_OPENSSL_VERSION_NUMBER >= 0x101fL) ||
> defined(OPENSSL_NO_DEPRECATED)
> >> > [...]
> >> > -#if defined(USE_THREAD) && (HA_OPENSSL_VERSION_NUMBER < 0x1010L)
> >> > +#if defined(USE_THREAD) && (HA_OPENSSL_VERSION_NUMBER < 0x1010L)
> && !defined(OPENSSL_NO_DEPRECATED)
> no idea what patch this is, but OPENSSL_NO_DEPRECATED should not be
> used anywhere. Always use OPENSSL_API_COMPAT.
> >> > [...]
> >> > -#if defined(USE_THREAD) && (HA_OPENSSL_VERSION_NUMBER < 0x1010L)
> >> > +#if defined(USE_THREAD) && (HA_OPENSSL_VERSION_NUMBER < 0x1010L)
> && !defined(OPENSSL_NO_DEPRECATED)
> >> > [...]
> >>
> >> I'm confused. This is not required in my environment for the build to
> >> succeed and I don't see any reason why HA_OPENSSL_VERSION_NUMBER would
> >> be smaller here? Can you elaborate why the HA_OPENSSL_VERSION_NUMBER
> >> comparison would fail to do its job in those comparisons?
> >
> >
> > what is the lowest openssl we support ?
> >
> > those callbacks are required if threads are used for non-deprecated
> builds and for early openssl versions like 1.0.0
> >>
> >>
> >> The X509_getm_ issue has been fixed by Rosen's patch [1], which is
> >> committed and backported.
> >>
> >> SSL_CTX_set_ecdh_auto issue is fixed by your patch (additional guard
> >> in ssl_sock.c) or by removing the existing guards and defining the
> >> same compatibility macro openssl uses [2] (as per the input from Willy
> >> and Emmanuel):
> >>
> >> #ifndef SSL_CTX_set_ecdh_auto
> >> #define SSL_CTX_set_ecdh_auto(dummy, onoff)  ((onoff) != 0)
> >> #endif
> >>
> >> I'd prefer the latter, which is what OpenSSL uses (when not using
> >> no-deprecated) and does not pollute the ssl_sock.c.
> >
> >
> > that's just perfect
> >
> >>
> >>
> >> Everything builds just fine after that for me (both master and 2.0),
> >> without any warnings. I also tried with threading disabled
> >> (USE_THREAD=).
> >>
> >> I will be sending the single SSL_CTX_set_ecdh_auto() fix shortly. Let
> >> me know what you think and if you believe something is missing for
> >> no-deprecated compatibility.
> >>
> >>
> >> FYI: to avoid rebuilding openssl each time with and without
> >> no-deprecate option, the same can be achieved when building haproxy by
> >> adding DEFINE="-DOPENSSL_API_COMPAT=0x1010L
> >> -DOPENSSL_NO_DEPRECATED" to the make command (maybe this can be useful
> >> in CI - I don't know anything about that).
>

I think we can take Lukas approach "not to reinvent the wheel, but take one
invented by openssl itself"

https://github.com/openssl/openssl/blob/bf4006a6f9be691ba6eef0e8629e63369a033ccf/include/openssl/crypto.h#L242-L246

:)


thank for the hint


> >
> >
> > yep, I'll have a look at that and will send patch for CI
> >
> >>
> >>
> >> Once we agree on a fix and commit it, we should definitely add a CI
> >> build testing this (with openssl 1.1.1). I disagree to test the build
> >> against openssl master, because the API may continually change during
> >> development (I mentioned this point in another conversation but I
> >> don't recall whether it was on ML or GH).
> >>
> >>
> >>
> >> thanks,
> >> lukas
> >>
> >>
> >> [1]
> https://github.com/haproxy/haproxy/commit/b3814c2ca8a8c28a890f8f50e0a35d5247222a12
> >> [2]
> https://github.com/openssl/openssl/blob/bf4006a6f9be691ba6eef0e8629e63369a033ccf/include/openssl/ssl.h#L1480
>

[PATCH] isolating ssl lib with rpath instead of LD_LIBRARY_PATH

[Илья Шипицин]

hello,

initially modyfing LD_LIBRARY_PATH seemed to be good, but it turned out
that other dynamically linked utilities also use ssl lib which is not
wanted.

using rpath in turn is more intelligent way of dynamic linking.

Cheers,
Ilya Shipitcin
From e9f95cc6fd8b61ee1a4aef05adade30fa72e5cd7 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Fri, 20 Dec 2019 23:55:01 +0500
Subject: [PATCH] BUILD: travis-ci: link with ssl libraries using rpath instead
 of LD_LIBRARY_PATH/DYLD_LIBRARY_PATH

modifying LD_LIBRARY_PATH/DYLD_LIBRARY_PATH also affects other utilities like curl
to avoid side effects let us use rpath for ssl library linking

Fixes #418
---
 .travis.yml | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index 9f1ec0195..6df39f52b 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -97,9 +97,7 @@ install:
 
 script:
   - make -C contrib/wurfl
-  - make -j3 CC=$CC V=1 TARGET=$TARGET $FLAGS DEBUG_CFLAGS="$DEBUG_CFLAGS" LDFLAGS="$LDFLAGS" 51DEGREES_SRC="$FIFTYONEDEGREES_SRC" EXTRA_OBJS="$EXTRA_OBJS"
-  - if [ "${TRAVIS_OS_NAME}" = "linux" ]; then export LD_LIBRARY_PATH="${HOME}/opt/lib:${LD_LIBRARY_PATH:-}"; fi
-  - if [ "${TRAVIS_OS_NAME}" = "osx" ]; then export DYLD_LIBRARY_PATH="${HOME}/opt/lib:${DYLD_LIBRARY_PATH:-}"; fi
+  - make -j3 CC=$CC V=1 TARGET=$TARGET $FLAGS DEBUG_CFLAGS="$DEBUG_CFLAGS" LDFLAGS="-L$SSL_LIB -Wl,-rpath,$SSL_LIB" 51DEGREES_SRC="$FIFTYONEDEGREES_SRC" EXTRA_OBJS="$EXTRA_OBJS"
   - ./haproxy -vv
   - if [ "${TRAVIS_OS_NAME}" = "linux" ]; then ldd haproxy; fi
   - if [ "${TRAVIS_OS_NAME}" = "osx" ]; then otool -L haproxy; fi
-- 
2.24.1

Re: PATCH: partially fix build if OpenSSL is built with no-deprecated option

[Илья Шипицин]

пт, 20 дек. 2019 г. в 22:39, Lukas Tribus :

> Hello Ilya,
>
>
>
> sorry about the delay ...
>
>
> On Wed, 27 Nov 2019 at 07:11, Илья Шипицин  wrote:
> >
> > -#if (HA_OPENSSL_VERSION_NUMBER >= 0x101fL)
> > +#if (HA_OPENSSL_VERSION_NUMBER >= 0x101fL) ||
> defined(OPENSSL_NO_DEPRECATED)
> > [...]
> > -#if defined(USE_THREAD) && (HA_OPENSSL_VERSION_NUMBER < 0x1010L)
> > +#if defined(USE_THREAD) && (HA_OPENSSL_VERSION_NUMBER < 0x1010L) &&
> !defined(OPENSSL_NO_DEPRECATED)
> > [...]
> > -#if defined(USE_THREAD) && (HA_OPENSSL_VERSION_NUMBER < 0x1010L)
> > +#if defined(USE_THREAD) && (HA_OPENSSL_VERSION_NUMBER < 0x1010L) &&
> !defined(OPENSSL_NO_DEPRECATED)
> > [...]
>
> I'm confused. This is not required in my environment for the build to
> succeed and I don't see any reason why HA_OPENSSL_VERSION_NUMBER would
> be smaller here? Can you elaborate why the HA_OPENSSL_VERSION_NUMBER
> comparison would fail to do its job in those comparisons?
>

what is the lowest openssl we support ?

those callbacks are required if threads are used for non-deprecated builds
and for early openssl versions like 1.0.0

>
> The X509_getm_ issue has been fixed by Rosen's patch [1], which is
> committed and backported.
>
> SSL_CTX_set_ecdh_auto issue is fixed by your patch (additional guard
> in ssl_sock.c) or by removing the existing guards and defining the
> same compatibility macro openssl uses [2] (as per the input from Willy
> and Emmanuel):
>
> #ifndef SSL_CTX_set_ecdh_auto
> #define SSL_CTX_set_ecdh_auto(dummy, onoff)  ((onoff) != 0)
> #endif
>
> I'd prefer the latter, which is what OpenSSL uses (when not using
> no-deprecated) and does not pollute the ssl_sock.c.
>

that's just perfect


>
> Everything builds just fine after that for me (both master and 2.0),
> without any warnings. I also tried with threading disabled
> (USE_THREAD=).
>
> I will be sending the single SSL_CTX_set_ecdh_auto() fix shortly. Let
> me know what you think and if you believe something is missing for
> no-deprecated compatibility.
>
>
> FYI: to avoid rebuilding openssl each time with and without
> no-deprecate option, the same can be achieved when building haproxy by
> adding DEFINE="-DOPENSSL_API_COMPAT=0x1010L
> -DOPENSSL_NO_DEPRECATED" to the make command (maybe this can be useful
> in CI - I don't know anything about that).
>

yep, I'll have a look at that and will send patch for CI


>
> Once we agree on a fix and commit it, we should definitely add a CI
> build testing this (with openssl 1.1.1). I disagree to test the build
> against openssl master, because the API may continually change during
> development (I mentioned this point in another conversation but I
> don't recall whether it was on ML or GH).
>
>
>
> thanks,
> lukas
>
>
> [1]
> https://github.com/haproxy/haproxy/commit/b3814c2ca8a8c28a890f8f50e0a35d5247222a12
> [2]
> https://github.com/openssl/openssl/blob/bf4006a6f9be691ba6eef0e8629e63369a033ccf/include/openssl/ssl.h#L1480
>

Re: [RFC PATCH] BUILD: ssl: improve SSL_CTX_set_ecdh_auto compatibility

[Илья Шипицин]

пт, 20 дек. 2019 г. в 22:47, Lukas Tribus :

> SSL_CTX_set_ecdh_auto() is not defined when OpenSSL 1.1.1 is compiled
> with the no-deprecated option. Remove existing, incomplete guards and
> add a compatibility macro in openssl-compat.h, just as OpenSSL does:
>
>
> https://github.com/openssl/openssl/blob/bf4006a6f9be691ba6eef0e8629e63369a033ccf/include/openssl/ssl.h#L1486
> ---
>
> Please wait for Ilya's comments before committing this patch.
>

Ack from me.


> thanks,
> -l
>
> ---
>  include/common/openssl-compat.h | 4 
>  src/ssl_sock.c  | 2 --
>  2 files changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/include/common/openssl-compat.h
> b/include/common/openssl-compat.h
> index 31971bd..72b4e2f 100644
> --- a/include/common/openssl-compat.h
> +++ b/include/common/openssl-compat.h
> @@ -374,5 +374,9 @@ static inline void EVP_PKEY_up_ref(EVP_PKEY *pkey)
>  #define BIO_meth_set_destroy(m, f) do { (m)->destroy = (f); } while (0)
>  #endif
>
> +#ifndef SSL_CTX_set_ecdh_auto
> +#define SSL_CTX_set_ecdh_auto(dummy, onoff)  ((onoff) != 0)
> +#endif
> +
>  #endif /* USE_OPENSSL */
>  #endif /* _COMMON_OPENSSL_COMPAT_H */
> diff --git a/src/ssl_sock.c b/src/ssl_sock.c
> index 00258b1..e4dd913 100644
> --- a/src/ssl_sock.c
> +++ b/src/ssl_sock.c
> @@ -5178,9 +5178,7 @@ int ssl_sock_prepare_ctx(struct bind_conf
> *bind_conf, struct ssl_bind_conf *ssl_
>   err && *err ? *err : "", curproxy->id,
> conf_curves, bind_conf->arg, bind_conf->file, bind_conf->line);
> cfgerr |= ERR_ALERT | ERR_FATAL;
> }
> -#if defined(SSL_CTX_set_ecdh_auto)
> (void)SSL_CTX_set_ecdh_auto(ctx, 1);
> -#endif
> }
>  #endif
>  #if defined(SSL_CTX_set_tmp_ecdh) && !defined(OPENSSL_NO_ECDH)
> --
> 2.7.4
>
>

Re: How to "unit" test HAProxy configurations (and HTTP servers in general)

[Илья Шипицин]

чт, 19 дек. 2019 г. в 01:07, Ciprian Dorin Craciun <
ciprian.crac...@gmail.com>:

> On Wed, Dec 18, 2019 at 8:23 PM Илья Шипицин  wrote:
> >> redirects are easy to test. using any framework.
> >
> > for example, jmeter (or any other http query tool)
> >
> >> * apply authentication for `admin`;
> >> * force some caching headers for `web`, `static` and `media`;
> >
> > same here. assert in jmeter would serve caching headers check
> >
> >> * apply some "sanity" checks to requests / responses (i.e. except
> >> admin, the rest should only serve `GET` requests);
> >> * deny any request that doesn't match a set of domains
> >
> > same here. just fire random query using jmeter
>
>
> JMeter, like Postman, don't seem very adequate tools for this purpose;
>  for starters they are too centered around a UI (especially for
> "editing" the tests).  (Although they have CLI runners, and perhaps
> their "tests collections" can be serialized in a file and put in a Git
> repository, I don't think one can easily use it with e.g. a custom
> editor, etc., like any "normal" configuration.)
>
> For example:
> * https://jmeter.apache.org/demos/ -- an XML file for JMeter JMX files
> used as "test plans";
> *
> https://github.com/Blazemeter/taurus/blob/master/examples/functional/postman-sample-collection.json
> -- a JSON file (random example) for Postman;
>
>
> The closest I could find something that resembles "usability" is
> Tavern, which at least has the advantage of YAML as opposed to "raw"
> XML or JSON:
> * https://github.com/taverntesting/tavern
> *
> https://github.com/taverntesting/tavern/blob/master/example/simple/test_server.tavern.yaml
>
>
> Really I would have expected to find a lightweight and generic Python
> or Ruby "framework" that allows one to write such tests in a more
> "programmatic" manner...  :(
>

python mechanize ?


>
> Ciprian.
>

Re: How to "unit" test HAProxy configurations (and HTTP servers in general)

[Илья Шипицин]

ср, 18 дек. 2019 г. в 22:34, Ciprian Dorin Craciun <
ciprian.crac...@gmail.com>:

> On Wed, Dec 18, 2019 at 6:47 PM Илья Шипицин  wrote:
> > you are talking about testing ACL. can you provide some example ?
>
>
> So let's assume I have a given HAProxy configuration, full of ACL's
> and rules, that apply certain "firewalling", authentication /
> authorization, "mangling" operations to the HTTP request (e.g. drop
> "bad-bots", update headers, redirects, routing to various backends,
> etc.).
>
> Now how can I test that the HAProxy configuration actually
> "implements" what it's proposes to?  I.e. how can I be sure that the
> rules are in the proper order, that no ACL's are missing, etc.
>
> My answer would be:  fire an HTTP request and see if it "does" what it
> should.  (Perhaps expose as HTTP headers some "state" values to help
> in checking things.)
>
>
>
>
> My concrete example would be this:  I find HAProxy wonderful for any
> non trivial HTTP deployment (and in fact anything "touched" by the
> Internet);  unfortunately the configuration language (with it's flat
> ACL's and request / response rules) is like "assembler" (as opposed to
> say Python).  Therefore I've written myself a HAProxy "configurator"
> in Python that based on simple Python code generates the full HAProxy
> configuration.
>
> For example:
>
>
> https://github.com/cipriancraciun/haproxy-configurator/blob/master/examples/example-01.py
>
> https://github.com/cipriancraciun/haproxy-configurator/blob/master/examples/_configs/example-01.cfg
>
> , the Python script is (hopefully) readable and clearly shows the
> intent of the resulting configuration:
> * redirect everything via HTTPS;
> * redirect `example.com` to `www.example.com`;
> * redirect `/admin/*` to `admin.example.com/admin/*`
> <http://admin.example.com/admin/*>, same for `/blog/*`;
>

redirects are easy to test. using any framework.

for example, jmeter (or any other http query tool)


> * apply authentication for `admin`;
> * force some caching headers for `web`, `static` and `media`;
>

same here. assert in jmeter would serve caching headers check


> * apply some "sanity" checks to requests / responses (i.e. except
> admin, the rest should only serve `GET` requests);
> * deny any request that doesn't match a set of domains
>

same here. just fire random query using jmeter


>
> The resulting file is around 639 lines, and (given how I've chosen to
> identify ACL's) is quite hard to "follow by hand".
>
> So my question now is how do I test it...  Fire HTTP requests at it!  :)
>
>
> I hope this gives everyone a glimpse into my use-case,
> Ciprian.
>

Re: How to "unit" test HAProxy configurations (and HTTP servers in general)

[Илья Шипицин]

ср, 18 дек. 2019 г. в 21:16, Ciprian Dorin Craciun <
ciprian.crac...@gmail.com>:

> Hello all!
>
> [First of all this question isn't only HAProxy specific, but can be
> extended to any HTTP server (including Apache, Nginx, and any other
> web application out there);  however I think it is especially
> important for HAProxy given how many HTTP-routing / mangling
> capabilities it has.]
>
> I have a quite complex HAProxy configuration and I want to make sure
> that while changing some ACL's and rules I don't break something else.
>
> Therefore I set out to find a tool that allows me to "unit test" an
> HTTP server (thus including HAProxy).  And to my surprise I didn't
> find one...  Obviously there are many "framework" unit test platforms
> out-there, each tailored for the underlying framework, such as Django,
> Flask, Rails, Spring, Go-lang, etc.;  however nothing "generic" that
> can test a web server by plain HTTP requests.
>
>
> So my question to the HAProxy community is if anyone knows / uses a
> generic HTTP unit testing framework.
>


I use openresty test framework (it works with nginx):
https://openresty.gitbooks.io/programming-openresty/content/testing/index.html
good question how to implement things using haproxy

you are talking about testing ACL. can you provide some example ?


>
> (I have already written a few custom Bash scripts that given an URL
> create a file on disk, and given it resides in a Git repository, I can
> easily `git diff ./tests/responses` to see if anything changed, but
> this is too "barbaric"...)  :)
>
> Thanks,
> Ciprian.
>
>

Re: [PATCH] openssl-compat: Fix getm_ defines

[Илья Шипицин]

вт, 17 дек. 2019 г. в 00:55, Rosen Penev :

> On Mon, Dec 16, 2019 at 10:21 AM Илья Шипицин 
> wrote:
> >
> >
> >
> > пн, 16 дек. 2019 г. в 22:40, Rosen Penev :
> >>
> >> On Mon, Dec 16, 2019 at 4:49 AM Lukas Tribus  wrote:
> >> >
> >> > Hello Rosen,
> >> >
> >> > > пн, 16 дек. 2019 г. в 12:07, Rosen Penev :
> >> > >>
> >> > >> LIBRESSL_VERSION_NUMBER evaluates to 0 under OpenSSL, making the
> condition
> >> > >> always true. Check for the define before checking it.
> >> >
> >> > I cannot find this in the openssl sources, not in master and not in
> >> > the 1.1.1 branch. Please clarify where this is defined.
> >> Compile with -Wundef. Missing macros evaluate to 0.
> >
> >
> > I checked haproxy source, it does not use such compiler flag. Any reason
> for introducing it ?
> >
> > if we want to make it first class citizen, maybe we should add it to
> proper Makefile ? or to our CI ?
> >
> > assuming "undefined macros may ACCIDENTLY become equal to 0" scares me
> You serious? This is basic C. Undefined macros always evaluate to 0.
>
>
indeed, you're right. I checked both gcc and clang.
unfortunately, I neither learned nor used that area of C preprocessor
specification before.


> -Wundef only warns about it.
> >
> >>
> >> >
> >> > The SSL compatibility layer is already complex enough and needs
> >> > continuous adjustments, we need to understand the reason for changes
> >> > very well. Fast fixes are continually coming back to hunt us.
> >> >
> >> >
> >> > On Mon, 16 Dec 2019 at 08:19, Илья Шипицин 
> wrote:
> >> > > please have a look at https://github.com/haproxy/haproxy/issues/367
> (it still misses germ part, I tried things like you send, but reg-tests
> fail. do you have travis-ci passed ?)
> >> > > also, there's a patch already sent, Lukas Tribus promised to review
> it
> >> >
> >> > Yeah, this one fell through the cracks. Give me a few days to catch
> up.
> >> >
> >> > Thanks,
> >> > Lukas
>

Re: [PATCH] openssl-compat: Fix getm_ defines

[Илья Шипицин]

пн, 16 дек. 2019 г. в 22:40, Rosen Penev :

> On Mon, Dec 16, 2019 at 4:49 AM Lukas Tribus  wrote:
> >
> > Hello Rosen,
> >
> > > пн, 16 дек. 2019 г. в 12:07, Rosen Penev :
> > >>
> > >> LIBRESSL_VERSION_NUMBER evaluates to 0 under OpenSSL, making the
> condition
> > >> always true. Check for the define before checking it.
> >
> > I cannot find this in the openssl sources, not in master and not in
> > the 1.1.1 branch. Please clarify where this is defined.
> Compile with -Wundef. Missing macros evaluate to 0.
>

I checked haproxy source, it does not use such compiler flag. Any reason
for introducing it ?

if we want to make it first class citizen, maybe we should add it to proper
Makefile ? or to our CI ?

assuming "undefined macros may ACCIDENTLY become equal to 0" scares me


> >
> > The SSL compatibility layer is already complex enough and needs
> > continuous adjustments, we need to understand the reason for changes
> > very well. Fast fixes are continually coming back to hunt us.
> >
> >
> > On Mon, 16 Dec 2019 at 08:19, Илья Шипицин  wrote:
> > > please have a look at https://github.com/haproxy/haproxy/issues/367
> (it still misses germ part, I tried things like you send, but reg-tests
> fail. do you have travis-ci passed ?)
> > > also, there's a patch already sent, Lukas Tribus promised to review it
> >
> > Yeah, this one fell through the cracks. Give me a few days to catch up.
> >
> > Thanks,
> > Lukas
>

Re: [PATCH] openssl-compat: Fix getm_ defines

[Илья Шипицин]

пн, 16 дек. 2019 г. в 22:42, Rosen Penev :

> LIBRESSL_VERSION_NUMBER evaluates to 0 under OpenSSL, making the condition
> always true. Check for the define before checking it.
>
> Signed-off-by: Rosen Penev 
> ---
>  include/common/openssl-compat.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/include/common/openssl-compat.h
> b/include/common/openssl-compat.h
> index 25102fbe3..c5029d133 100644
> --- a/include/common/openssl-compat.h
> +++ b/include/common/openssl-compat.h
> @@ -278,7 +278,7 @@ static inline void EVP_PKEY_up_ref(EVP_PKEY *pkey)
>  #define TLSEXT_signature_ecdsa  3
>  #endif
>
> -#if (OPENSSL_VERSION_NUMBER < 0x1010L) || (LIBRESSL_VERSION_NUMBER <
> 0x2070L)
> +#if (HA_OPENSSL_VERSION_NUMBER < 0x101fL) && (LIBRESSL_VERSION_NUMBER
> < 0x207fL)
>

assuming "&& (LIBRESSL_VERSION_NUMBER < 0x207fL)" part ... it is only
relevant for LibreSSL, right ?
if so, should we leave just second part and omit first ?



>  #define X509_getm_notBefore X509_get_notBefore
>  #define X509_getm_notAfter  X509_get_notAfter
>  #endif
> --
> 2.23.0
>
>
>

Re: [PATCH] openssl-compat: Fix getm_ defines

[Илья Шипицин]

also, BoringSSL fails after applying your patch

https://travis-ci.com/chipitsine/haproxy/jobs/267601286

пн, 16 дек. 2019 г. в 12:07, Rosen Penev :

> LIBRESSL_VERSION_NUMBER evaluates to 0 under OpenSSL, making the condition
> always true. Check for the define before checking it.
>
> Signed-off-by: Rosen Penev 
> ---
>  include/common/openssl-compat.h | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/include/common/openssl-compat.h
> b/include/common/openssl-compat.h
> index 25102fbe3..8b1630110 100644
> --- a/include/common/openssl-compat.h
> +++ b/include/common/openssl-compat.h
> @@ -278,7 +278,8 @@ static inline void EVP_PKEY_up_ref(EVP_PKEY *pkey)
>  #define TLSEXT_signature_ecdsa  3
>  #endif
>
> -#if (OPENSSL_VERSION_NUMBER < 0x1010L) || (LIBRESSL_VERSION_NUMBER <
> 0x2070L)
> +#if (OPENSSL_VERSION_NUMBER < 0x1010L) || \
> +   (defined(LIBRESSL_VERSION_NUMBER) && (LIBRESSL_VERSION_NUMBER <
> 0x2070L))
>  #define X509_getm_notBefore X509_get_notBefore
>  #define X509_getm_notAfter  X509_get_notAfter
>  #endif
> --
> 2.23.0
>
>
>

Re: [PATCH] openssl-compat: Fix getm_ defines

[Илья Шипицин]

пн, 16 дек. 2019 г. в 12:47, William Lallemand :

> Hello Rosen,
>
> On Sun, Dec 15, 2019 at 11:04:37PM -0800, Rosen Penev wrote:
> > -#if (OPENSSL_VERSION_NUMBER < 0x1010L) || (LIBRESSL_VERSION_NUMBER
> < 0x2070L)
> > +#if (OPENSSL_VERSION_NUMBER < 0x1010L) || \
> > + (defined(LIBRESSL_VERSION_NUMBER) && (LIBRESSL_VERSION_NUMBER <
> 0x2070L))
> >
>
> It's probably cleaner to use the HA_OPENSSL_VERSION_NUMBER function and
> something like this:
>

@Rosen Penev 

there was some discussion whether to trust OPENSSL_VERSION_NUMBER or not.
LibreSSL pollutes that macro, it sets it to 2.0.0

so ... new macro was introduced instead: HA_OPENSSL_VERSION_NUMBER

https://github.com/haproxy/haproxy/blob/master/include/common/openssl-compat.h#L27-L36


>
>#if (HA_OPENSSL_VERSION_NUMBER < 0x101fL) &&
> (LIBRESSL_VERSION_NUMBER < 0x207fL)
>
> --
> William Lallemand
>
>

Re: [PATCH] openssl-compat: Fix getm_ defines

[Илья Шипицин]

hello,

seems OpenWRT guys here :-)

please have a look at https://github.com/haproxy/haproxy/issues/367 (it
still misses germ part, I tried things like you send, but reg-tests fail.
do you have travis-ci passed ?)
also, there's a patch already sent, Lukas Tribus promised to review it

пн, 16 дек. 2019 г. в 12:07, Rosen Penev :

> LIBRESSL_VERSION_NUMBER evaluates to 0 under OpenSSL, making the condition
> always true. Check for the define before checking it.
>
> Signed-off-by: Rosen Penev 
> ---
>  include/common/openssl-compat.h | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/include/common/openssl-compat.h
> b/include/common/openssl-compat.h
> index 25102fbe3..8b1630110 100644
> --- a/include/common/openssl-compat.h
> +++ b/include/common/openssl-compat.h
> @@ -278,7 +278,8 @@ static inline void EVP_PKEY_up_ref(EVP_PKEY *pkey)
>  #define TLSEXT_signature_ecdsa  3
>  #endif
>
> -#if (OPENSSL_VERSION_NUMBER < 0x1010L) || (LIBRESSL_VERSION_NUMBER <
> 0x2070L)
> +#if (OPENSSL_VERSION_NUMBER < 0x1010L) || \
> +   (defined(LIBRESSL_VERSION_NUMBER) && (LIBRESSL_VERSION_NUMBER <
> 0x2070L))
>  #define X509_getm_notBefore X509_get_notBefore
>  #define X509_getm_notAfter  X509_get_notAfter
>  #endif
> --
> 2.23.0
>
>
>

adding 0RTT to some reg test ?

[Илья Шипицин]

hello,

what do you think if we add something like that ?

--- a/reg-tests/ssl/wrong_ctx_storage.vtc
+++ b/reg-tests/ssl/wrong_ctx_storage.vtc
@@ -30,7 +30,7 @@ haproxy h1 -conf {
   listen frt
 mode http
 ${no-htx} option http-use-htx
-bind "fd@${frt}" ssl crt ${testdir}/common.pem
+bind "fd@${frt}" allow-0rtt ssl crt ${testdir}/common.pem
 http-request redirect location /
 } -start

-- 
2.23.0

Re: [PATCH] CLEANUP: dns: resolution can never be null

[Илья Шипицин]

indeed it is checked earlier

https://github.com/haproxy/haproxy/blob/master/src/dns.c#L1574

чт, 28 нояб. 2019 г. в 12:18, William Dauchy :

> On Thu, Nov 28, 2019 at 11:10:34AM +0500, Илья Шипицин wrote:
> > Willy thinks it should be "eb" instead of "res"
>
> yup I saw that comment but I don't get why as it is tested above.
>
> --
> William
>

Re: [PATCH] CLEANUP: dns: resolution can never be null

[Илья Шипицин]

Willy thinks it should be "eb" instead of "res"


https://github.com/haproxy/haproxy/issues/349#issuecomment-548241746

On Thu, Nov 28, 2019, 3:33 AM William Dauchy  wrote:

> `eb` being tested above, `res` cannot be null, so the condition is
> not needed and introduces potential dead code.
>
> also fix a typo in associated comment
>
> This should fix issue #349
>
> Reported-by: Илья Шипицин 
> Signed-off-by: William Dauchy 
> ---
>  src/dns.c | 7 +--
>  1 file changed, 1 insertion(+), 6 deletions(-)
>
> diff --git a/src/dns.c b/src/dns.c
> index 6e726bf9..c52f1cab 100644
> --- a/src/dns.c
> +++ b/src/dns.c
> @@ -1577,13 +1577,8 @@ static void dns_resolve_recv(struct dgram_conn
> *dgram)
> continue;
> }
>
> -   /* known query id means a resolution in prgress */
> +   /* known query id means a resolution in progress */
> res = eb32_entry(eb, struct dns_resolution, qid);
> -   if (!res) {
> -   ns->counters.outdated++;
> -   continue;
> -   }
> -
> /* number of responses received */
> res->nb_responses++;
>
> --
> 2.24.0
>
>

Re: PATCH: partially fix build if OpenSSL is built with no-deprecated option

[Илья Шипицин]

ср, 27 нояб. 2019 г. в 05:02, Lukas Tribus :

> Hello,
>
> On Tue, Nov 26, 2019 at 10:50 PM Илья Шипицин 
> wrote:
> >
> > Hello,
> >
> > I resolved   `CRYPTO_set_id_callback', `ERR_remove_state',
> `SSL_CTX_set_ecdh_auto' issues.
> >
> >
> > the following two will be addressed later:  `X509_get_notBefore',
> `X509_get_notAfter'
>
> I'm not sure if matching OPENSSL_NO_DEPRECATED is a good idea, for
> compatibility reasons (what's deprecated in one release isn't in
> another). It is also not immediately clear to me if those old code
> paths are needed by older openssl or not (or only libressl)?
>
>
> Please allow *ample* time to review and test these patches before commit.
>


sure.
I will send v2 after your feedback

(also, I'm ok with notes from Tim and Willy (regarding changing commit
message and moving 'SSL_CTX_set_ecdh_auto'' to openssl_compat.h)


>
>
>
> Thanks,
>
> Lukas
>

PATCH: partially fix build if OpenSSL is built with no-deprecated option

[Илья Шипицин]

Hello,

I resolved   `CRYPTO_set_id_callback', `ERR_remove_state',
`SSL_CTX_set_ecdh_auto' issues.


the following two will be addressed later:  `X509_get_notBefore',
`X509_get_notAfter'


Cheers,

Ilya Shipitsin
From 158e07f29cbd40c7cd159330ffd748cb1b926647 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Wed, 27 Nov 2019 02:44:58 +0500
Subject: [PATCH] CLEANUP: partially resolve #367

if OpenSSL is built with no-deprecated mode, some functions are not available.
however, we keep those functions for LibreSSL when appropriate
---
 include/common/openssl-compat.h | 2 +-
 src/ssl_sock.c  | 6 --
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/include/common/openssl-compat.h b/include/common/openssl-compat.h
index 00395d3e7..2c0cac29e 100644
--- a/include/common/openssl-compat.h
+++ b/include/common/openssl-compat.h
@@ -213,7 +213,7 @@ static inline void EVP_PKEY_up_ref(EVP_PKEY *pkey)
  * 1.1.0 and does nothing anymore. Let's simply silently kill
  * it.
  */
-#if (HA_OPENSSL_VERSION_NUMBER >= 0x101fL)
+#if (HA_OPENSSL_VERSION_NUMBER >= 0x101fL) || defined(OPENSSL_NO_DEPRECATED)
 #undef  ERR_remove_state
 #define ERR_remove_state(x)
 #endif
diff --git a/src/ssl_sock.c b/src/ssl_sock.c
index fc7109f58..2834c10f8 100644
--- a/src/ssl_sock.c
+++ b/src/ssl_sock.c
@@ -308,7 +308,7 @@ static int ha_ssl_free(BIO *data)
 }
 
 
-#if defined(USE_THREAD) && (HA_OPENSSL_VERSION_NUMBER < 0x1010L)
+#if defined(USE_THREAD) && (HA_OPENSSL_VERSION_NUMBER < 0x1010L) && !defined(OPENSSL_NO_DEPRECATED)
 
 static HA_RWLOCK_T *ssl_rwlocks;
 
@@ -5046,7 +5046,9 @@ int ssl_sock_prepare_ctx(struct bind_conf *bind_conf, struct ssl_bind_conf *ssl_
 			 NULL);
 
 		if (ecdhe == NULL) {
+#if defined(SSL_CTX_set_ecdh_auto)
 			(void)SSL_CTX_set_ecdh_auto(ctx, 1);
+#endif
 			return cfgerr;
 		}
 #else
@@ -11055,7 +11057,7 @@ static void __ssl_sock_init(void)
 	}
 #endif
 
-#if defined(USE_THREAD) && (HA_OPENSSL_VERSION_NUMBER < 0x1010L)
+#if defined(USE_THREAD) && (HA_OPENSSL_VERSION_NUMBER < 0x1010L) && !defined(OPENSSL_NO_DEPRECATED)
 	ssl_locking_init();
 #endif
 #if (HA_OPENSSL_VERSION_NUMBER >= 0x1000200fL && !defined OPENSSL_NO_TLSEXT && !defined OPENSSL_IS_BORINGSSL)
-- 
2.23.0

Re: HAProxy 2.0.10 and 2.1.0 RPM's

[Илья Шипицин]

ср, 27 нояб. 2019 г. в 01:10, Russell Eason :

> Hello,
>
> Fedora upstream added it
> https://src.fedoraproject.org/rpms/haproxy/c/45c57ba71174f308a5f59569bac0598bb31ef767
> , and can be seen as far back as F24 here
> https://src.fedoraproject.org/rpms/haproxy/blob/f24/f/haproxy.spec . LUA
> support is in the RHEL 8 version of HAProxy, but not in 7 (yet?).
>

I really stuck on that.
haproxy needs Lua-5.3, there's no good for CentOS 7


>
> Russell
>
> On Tue, Nov 26, 2019 at 3:02 PM William Lallemand 
> wrote:
>
>> On Tue, Nov 26, 2019 at 08:33:41PM +0100, Julien Pivotto wrote:
>> > Dear HAProxy Community,
>> >
>> > I have started building HAProxy 2.x packages for CentOS.
>> >
>> > It includes HAProxy 2.0.10 and 2.1.0.
>> >
>> > You can find them here:
>> > https://copr.fedorainfracloud.org/coprs/roidelapluie/haproxy/
>> >
>> > https://github.com/roidelapluie/haproxy-rpm
>> > which is based on https://git.centos.org/rpms/rh-haproxy18-haproxy
>> >
>> > Repo config:
>> >
>> https://copr.fedorainfracloud.org/coprs/roidelapluie/haproxy/repo/epel-7/roidelapluie-haproxy-epel-7.repo
>> >
>> > Copr is the Fedora public tool to build packages. Build logs are public,
>> > as well as source RPM's etc. So you are free to review it.
>> >
>>
>> Hi Julien,
>>
>> Thanks for your work, we really lack up-to-date packages for centos/rhel,
>> that's a relief people are still trying to update them. :-)
>>
>> We don't really have an official rhel/centos package for HAProxy, but
>> there are
>> multiple ones. I think it could be really great if people interested in
>> HAProxy
>> for redhat-based distribution could work together to maintain one
>> "official"
>> repository like we have for debian/ubuntu. (http://haproxy.debian.net)
>>
>> We already knew about these ones:
>>
>> - http://haproxy.hongens.nl/
>> - https://repo.ius.io/7/x86_64/packages/h/
>>
>> But none of them are up to date with the latest version of their branches
>> .
>>
>> I didn't knew about Copr, it looks like a PPA-like for redhat from what I
>> can
>> see, it could be a really great platform to do that.
>>
>> Thanks,
>>
>> --
>> William Lallemand
>>
>>

Re: HAProxy 2.0.10 and 2.1.0 RPM's

[Илья Шипицин]

ср, 27 нояб. 2019 г. в 01:03, William Lallemand :

> On Tue, Nov 26, 2019 at 08:33:41PM +0100, Julien Pivotto wrote:
> > Dear HAProxy Community,
> >
> > I have started building HAProxy 2.x packages for CentOS.
> >
> > It includes HAProxy 2.0.10 and 2.1.0.
> >
> > You can find them here:
> > https://copr.fedorainfracloud.org/coprs/roidelapluie/haproxy/
> >
> > https://github.com/roidelapluie/haproxy-rpm
> > which is based on https://git.centos.org/rpms/rh-haproxy18-haproxy
> >
> > Repo config:
> >
> https://copr.fedorainfracloud.org/coprs/roidelapluie/haproxy/repo/epel-7/roidelapluie-haproxy-epel-7.repo
> >
> > Copr is the Fedora public tool to build packages. Build logs are public,
> > as well as source RPM's etc. So you are free to review it.
> >
>
> Hi Julien,
>
> Thanks for your work, we really lack up-to-date packages for centos/rhel,
> that's a relief people are still trying to update them. :-)
>
> We don't really have an official rhel/centos package for HAProxy, but
> there are
> multiple ones. I think it could be really great if people interested in
> HAProxy
> for redhat-based distribution could work together to maintain one
> "official"
> repository like we have for debian/ubuntu. (http://haproxy.debian.net)
>
> We already knew about these ones:
>
> - http://haproxy.hongens.nl/
> - https://repo.ius.io/7/x86_64/packages/h/
>
> But none of them are up to date with the latest version of their branches .
>
> I didn't knew about Copr, it looks like a PPA-like for redhat from what I
> can
> see, it could be a really great platform to do that.
>

exactly


>
> Thanks,
>
> --
> William Lallemand
>
>

Re: HAProxy 2.0.10 and 2.1.0 RPM's

[Илья Шипицин]

thank for the rpm

it was faster than I did ))

any plans to add Lua ?

ср, 27 нояб. 2019 г. в 00:36, Julien Pivotto :

> Dear HAProxy Community,
>
> I have started building HAProxy 2.x packages for CentOS.
>
> It includes HAProxy 2.0.10 and 2.1.0.
>
> You can find them here:
> https://copr.fedorainfracloud.org/coprs/roidelapluie/haproxy/
>
> https://github.com/roidelapluie/haproxy-rpm
> which is based on https://git.centos.org/rpms/rh-haproxy18-haproxy
>
> Repo config:
>
> https://copr.fedorainfracloud.org/coprs/roidelapluie/haproxy/repo/epel-7/roidelapluie-haproxy-epel-7.repo
>
> Copr is the Fedora public tool to build packages. Build logs are public,
> as well as source RPM's etc. So you are free to review it.
>
> --
>  (o-Julien Pivotto
>  //\Open-Source Consultant
>  V_/_   Inuits - https://www.inuits.eu
>

Re: [PATCH] MINOR: contrib/prometheus-exporter: allow to select the exported metrics

[Илья Шипицин]

чт, 21 нояб. 2019 г. в 15:18, William Dauchy :

> On Thu, Nov 21, 2019 at 03:09:30PM +0500, Илья Шипицин wrote:
> > I understand. However, those patches add complexity (which might be moved
> > to another dedicated tool)
>
> those patch makes sense for heavy haproxy instances. As you might have
> seen above, we are talking about > 130MB of data. So for a full scraping
> every 60s or less, this is not realistic. Even the data loading might
> take too much time. We had cases where loading the data on exporter side
> was taking more time than the frequency of scraping, generating a
> snowball effect.
> It's a good pratice to avoid exporting data you know
> you won't use instead of: scraping -> deleting -> aggregate
> In our case we do not use most of the server metrics. It represents a
> factor of 10 in terms of exported data.
>

yep. I did see 130mb and I was impressed.


>
> --
> William
>

Re: [PATCH] MINOR: contrib/prometheus-exporter: allow to select the exported metrics

[Илья Шипицин]

чт, 21 нояб. 2019 г. в 15:07, William Dauchy :

> On Thu, Nov 21, 2019 at 03:00:02PM +0500, Илья Шипицин wrote:
> > is it common thing in Prometheus world to perform cascade export ? like
> > "exporter" --> "filter out" --> "aggregate" --> "deliver to prometheus"
> > in order to keep things simple and not to push everything into single
> tool
>
> no, the best practice is often to keep the original data in a local
> prometheus, and aggregate (quite often you use another instance for
> aggregation)
> But here those patch are about dropping the data you know you will never
> use.
>

I understand. However, those patches add complexity (which might be moved
to another dedicated tool)


> --
> William
>

Re: [PATCH] MINOR: contrib/prometheus-exporter: allow to select the exported metrics

[Илья Шипицин]

btw, side question...

is it common thing in Prometheus world to perform cascade export ? like
"exporter" --> "filter out" --> "aggregate" --> "deliver to prometheus"
in order to keep things simple and not to push everything into single tool

чт, 21 нояб. 2019 г. в 14:49, Christopher Faulet :

> Le 20/11/2019 à 21:23, William Dauchy a écrit :
> > Hi Christopher,
> >
> > On Wed, Nov 20, 2019 at 02:56:28PM +0100, Christopher Faulet wrote:
> >> Nice, Thanks for your feedback. It is merged now. And I'm on the
> backports
> >> for the 2.0.
> >
> > You apparently forgot to backport
> > commit 0d1c2a65e8370a770d01 (MINOR: stats: Report max times in addition
> of the averages for sessions)
> >
> > 2.0 tree does not build anymore because ST_F_QT_MAX is not defined.
> >
>
> Damned ! You're right. I'm sorry. It was backported now. Thanks !
>
> --
> Christopher Faulet
>
>

Re: travis-ci: should we drop openssl-1.1.0 and replace it with 3.0 ?

[Илья Шипицин]

yep, 3.0 stands for openssl master branch.
the point is to catch incompatibilities before it is released.

вт, 19 нояб. 2019 г. в 22:51, Gibson, Brian (IMS) :

> Maybe after they stop security fixes we can drop 1.1.0.  I know there are
> many distributions still in support that use this branch.  3.0 doesn’t
> exist yet, and won’t until later in 2020 which is unfortunate since that
> means there will be no FIPS validated branch for several months.
>
>
>
> *From:* Илья Шипицин [mailto:chipits...@gmail.com]
> *Sent:* Tuesday, November 19, 2019 12:48 PM
> *To:* HAProxy 
> *Subject:* Re: travis-ci: should we drop openssl-1.1.0 and replace it
> with 3.0 ?
>
>
>
> well, we can actually build bigger matrix by adding builds. I just want to
> save some electricity on non needed builds.
>
>
>
> вт, 19 нояб. 2019 г. в 22:41, Илья Шипицин :
>
> hello,
>
>
>
> https://www.openssl.org/source/ says "The 1.1.0 series is currently only
> receiving security fixes and will go out of support on 11th September 2019"
>
>
>
>
>
> what if we drop it ? and replace with 3.0 ?
>
>
>
> cheers,
>
> Ilya Shipitcin
>
>
> --
>
> Information in this e-mail may be confidential. It is intended only for
> the addressee(s) identified above. If you are not the addressee(s), or an
> employee or agent of the addressee(s), please note that any dissemination,
> distribution, or copying of this communication is strictly prohibited. If
> you have received this e-mail in error, please notify the sender of the
> error.
>

Re: travis-ci: should we drop openssl-1.1.0 and replace it with 3.0 ?

[Илья Шипицин]

well, we can actually build bigger matrix by adding builds. I just want to
save some electricity on non needed builds.

вт, 19 нояб. 2019 г. в 22:41, Илья Шипицин :

> hello,
>
> https://www.openssl.org/source/ says "The 1.1.0 series is currently only
> receiving security fixes and will go out of support on 11th September 2019"
>
>
> what if we drop it ? and replace with 3.0 ?
>
> cheers,
> Ilya Shipitcin
>

travis-ci: should we drop openssl-1.1.0 and replace it with 3.0 ?

[Илья Шипицин]

hello,

https://www.openssl.org/source/ says "The 1.1.0 series is currently only
receiving security fixes and will go out of support on 11th September 2019"


what if we drop it ? and replace with 3.0 ?

cheers,
Ilya Shipitcin

http-buffer-request details

[Илья Шипицин]

hello,

how is that supposed to work ?

https://github.com/haproxy/haproxy/blob/master/doc/configuration.txt#L6225

does it buffer the entire body ? does it use memory / hdd for buffering ?

how are those buffers allocated ? what if I do not have a lot of RAM ?

thanks,
Ilya Shipitcin

Re: [PATCH] BUG/MINOR: init: fix set-dumpable when using uid/gid

[Илья Шипицин]

вт, 19 нояб. 2019 г. в 14:15, William Dauchy :

> in mworker mode used with uid/gid settings, it was not possible to get
> a coredump despite the set-dumpable option.
> indeed prctl(2) manual page specifies the dumpable attribute is reverted
> to `/proc/sys/fs/suid_dumpable` in a few conditions such as process
> effective user and group are changed.
>

small question.

`/proc/sys/fs/suid_dumpable` is linux specific. will it work under freebsd,
openbsd ? windows ?
also, linux might not mount that filesystem. will it work ?



>
> this patch moves the whole set-dumpable logic before the polling code in
> order to catch all possible cases where we could have changed the
> uid/gid. It however does not cover the possible segfault at startup.
>
> this patch should be backported in 2.0.
>
> Signed-off-by: William Dauchy 
>
> ---
>
> Hi Willy,
>
> Here is the backport for haproxy-20 tree.
>
> Thanks,
>
> William
>
> ---
>  src/haproxy.c | 50 +-
>  1 file changed, 25 insertions(+), 25 deletions(-)
>
> diff --git a/src/haproxy.c b/src/haproxy.c
> index fa3fbad4..a0e630df 100644
> --- a/src/haproxy.c
> +++ b/src/haproxy.c
> @@ -2946,31 +2946,6 @@ int main(int argc, char **argv)
> }
> }
>
> -   /* try our best to re-enable core dumps depending on system
> capabilities.
> -* What is addressed here :
> -*   - remove file size limits
> -*   - remove core size limits
> -*   - mark the process dumpable again if it lost it due to
> user/group
> -*/
> -   if (global.tune.options & GTUNE_SET_DUMPABLE) {
> -   limit.rlim_cur = limit.rlim_max = RLIM_INFINITY;
> -
> -#if defined(RLIMIT_FSIZE)
> -   if (setrlimit(RLIMIT_FSIZE, ) == -1)
> -   ha_warning("[%s.main()] Failed to set the raise
> the maximum file size.\n", argv[0]);
> -#endif
> -
> -#if defined(RLIMIT_CORE)
> -   if (setrlimit(RLIMIT_CORE, ) == -1)
> -   ha_warning("[%s.main()] Failed to set the raise
> the core dump size.\n", argv[0]);
> -#endif
> -
> -#if defined(USE_PRCTL)
> -   if (prctl(PR_SET_DUMPABLE, 1, 0, 0, 0) == -1)
> -   ha_warning("[%s.main()] Failed to set the dumpable
> flag, no core will be dumped.\n", argv[0]);
> -#endif
> -   }
> -
> /* check ulimits */
> limit.rlim_cur = limit.rlim_max = 0;
> getrlimit(RLIMIT_NOFILE, );
> @@ -3253,6 +3228,31 @@ int main(int argc, char **argv)
> fork_poller();
> }
>
> +   /* try our best to re-enable core dumps depending on system
> capabilities.
> +* What is addressed here :
> +*   - remove file size limits
> +*   - remove core size limits
> +*   - mark the process dumpable again if it lost it due to
> user/group
> +*/
> +   if (global.tune.options & GTUNE_SET_DUMPABLE) {
> +   limit.rlim_cur = limit.rlim_max = RLIM_INFINITY;
> +
> +#if defined(RLIMIT_FSIZE)
> +   if (setrlimit(RLIMIT_FSIZE, ) == -1)
> +   ha_warning("[%s.main()] Failed to set the raise
> the maximum file size.\n", argv[0]);
> +#endif
> +
> +#if defined(RLIMIT_CORE)
> +   if (setrlimit(RLIMIT_CORE, ) == -1)
> +   ha_warning("[%s.main()] Failed to set the raise
> the core dump size.\n", argv[0]);
> +#endif
> +
> +#if defined(USE_PRCTL)
> +   if (prctl(PR_SET_DUMPABLE, 1, 0, 0, 0) == -1)
> +   ha_warning("[%s.main()] Failed to set the dumpable
> flag, no core will be dumped.\n", argv[0]);
> +#endif
> +   }
> +
> global.mode &= ~MODE_STARTING;
> /*
>  * That's it : the central polling loop. Run until we stop.
> --
> 2.24.0
>
>
>

Re: arm64 builds?

[Илья Шипицин]

пн, 4 нояб. 2019 г. в 17:49, Willy Tarreau :

> Hi Ilya,
>
> On Mon, Nov 04, 2019 at 12:18:49AM +0500,  ??? wrote:
> > hello,
> >
> > should we switch some builds to arm64?
> >
> > https://blog.travis-ci.com/2019-10-07-multi-cpu-architecture-support
>
> Ah that's interesting! I frequently build for arm/arm64 but I agree
> it would be nice to have this done more frequently. The two main points
> I'm seeing are :
>   - unsigned chars, which occasionally trigger a warning somewhere
>   - non-x86 build, which can occasionally trigger a build error if
> we accidently rely on an arch-specific function
>
> In fact I would even suggest that we build for arm instead of arm64 so
> that we switch to 32 bits at the same time and have an opportunity to
> detect long vs int vs uint64t vs pointer vs size_t issues (typically
> places where printf uses "%lu" without a cast for uint64_t).
>


only arm64 are available.
we can try arm using cross build, for example. is it common way to use
cross build for building haproxy ?


>
> Thanks,
> Willy
>

arm64 builds?

[Илья Шипицин]

hello,

should we switch some builds to arm64?

https://blog.travis-ci.com/2019-10-07-multi-cpu-architecture-support

thanks,
Ilya Shipitsin

interoperation between SCVMM and HAProxy dataplane api

[Илья Шипицин]

hi,

is anybody running things like
https://www.poppelgaard.com/microsoft-scvmm-2012citrix-netscaler-o ?

I mean using SCVMM in so called "services management", i.e. dataplane api ?

thanks,
Ilya Shipitsin

google summer of code 2020 ?

[Илья Шипицин]

hello,

are there any plans to participate GSoC 2020 ?

thanks,
Ilya Shipitsin

Re: cygwin builds

[Илья Шипицин]

BUILD: CI: comment out cygwin build, upgrade various ssl libraries

cirrus ci builds are now limited to branches master, next
travis-ci images are upgraded to ubuntu bionic
cygwin builds are temporarily disabled on travis-ci
(maybe someone will figure out how to fix them, here's link

https://travis-ci.community/t/cygwin-issue-cygheap-base-mismatch-detected/5359/2
)

openssl upgraded to 1.0.2t, 1.1.0l, 1.1.1d
libressl are upgraded (2.9.2, 2.8.3, 2.7.5)  --> (3.0.2, 2.9.2, 2.8.3)

вс, 27 окт. 2019 г. в 19:29, Илья Шипицин :

> hi,
>
> I spent few days getting travis-ci / cygwin work.
> no luck
>
> I'm going to disable them unless someone knows how to resolve that
>
>
> thanks,
> Ilya Shipitsin
>
From a4d00f6a1c50420e2045e9ccd509072190b1cad0 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Sun, 27 Oct 2019 20:16:29 +0500
Subject: [PATCH] BUILD: CI: comment out cygwin build, upgrade various ssl
 libraries

cirrus ci builds are now limited to branches master, next
travis-ci images are upgraded to ubuntu bionic
cygwin builds are temporarily disabled on travis-ci
(maybe someone will figure out how to fix them, here's link
https://travis-ci.community/t/cygwin-issue-cygheap-base-mismatch-detected/5359/2 )

openssl upgraded to 1.0.2t, 1.1.0l, 1.1.1d
libressl are upgraded (2.9.2, 2.8.3, 2.7.5)  --> (3.0.2, 2.9.2, 2.8.3)
---
 .cirrus.yml |  2 ++
 .travis.yml | 30 +++---
 2 files changed, 17 insertions(+), 15 deletions(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index 5d3870954..e4278c2ce 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -1,6 +1,7 @@
 FreeBSD_task:
   freebsd_instance:
 image: freebsd-12-0-release-amd64
+  only_if: $CIRRUS_BRANCH =~ 'master|next'
   env:
 ASSUME_ALWAYS_YES: TRUE # required for unattended "pkg" invocation
   script:
@@ -15,6 +16,7 @@ FreeBSD_task:
 centos_6_task:
   container:
 image: centos:centos6
+  only_if: $CIRRUS_BRANCH =~ 'master|next'
   script:
 - yum install -q -y gcc git openssl-devel pcre-devel epel-release
 - yum install -q -y python34
diff --git a/.travis.yml b/.travis.yml
index f3fe008fc..9f1ec0195 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,7 +1,7 @@
 # build status appears on https://travis-ci.com/haproxy/haproxy
 
 sudo: required
-dist: xenial
+dist: bionic
 
 language: c
 
@@ -32,15 +32,15 @@ matrix:
   - os: linux-ppc64le
 if: type == cron
 compiler: gcc
-env: TARGET=linux-glibc OPENSSL_VERSION=1.0.2r LABEL="linux-ppc64le"
+env: TARGET=linux-glibc OPENSSL_VERSION=1.0.2t LABEL="linux-ppc64le"
   - os: linux
 if: type != cron
 compiler: clang
-env: TARGET=linux-glibc OPENSSL_VERSION=1.1.1c
+env: TARGET=linux-glibc OPENSSL_VERSION=1.1.1d
   - os: linux
 if: type == cron
 compiler: clang
-env: TARGET=linux-glibc OPENSSL_VERSION=1.1.1c COVERITY_SCAN_PROJECT_NAME="Haproxy" COVERITY_SCAN_BRANCH_PATTERN="*" COVERITY_SCAN_NOTIFICATION_EMAIL="chipits...@gmail.com" COVERITY_SCAN_BUILD_COMMAND="make CC=clang TARGET=$TARGET $FLAGS 51DEGREES_SRC=$FIFTYONEDEGREES_SRC"
+env: TARGET=linux-glibc OPENSSL_VERSION=1.1.1d COVERITY_SCAN_PROJECT_NAME="Haproxy" COVERITY_SCAN_BRANCH_PATTERN="*" COVERITY_SCAN_NOTIFICATION_EMAIL="chipits...@gmail.com" COVERITY_SCAN_BUILD_COMMAND="make CC=clang TARGET=$TARGET $FLAGS 51DEGREES_SRC=$FIFTYONEDEGREES_SRC"
 script:
   - |
 if [ ! -z ${COVERITY_SCAN_TOKEN+x} ]; then
@@ -50,19 +50,19 @@ matrix:
   - os: linux
 if: type == cron
 compiler: clang
-env: TARGET=linux-glibc OPENSSL_VERSION=1.1.0j FIFTYONEDEGREES_SRC="contrib/51d/src/trie"
+env: TARGET=linux-glibc OPENSSL_VERSION=1.1.0l FIFTYONEDEGREES_SRC="contrib/51d/src/trie"
   - os: linux
 if: type != cron
 compiler: clang
-env: TARGET=linux-glibc LIBRESSL_VERSION=2.9.2
+env: TARGET=linux-glibc LIBRESSL_VERSION=3.0.2
   - os: linux
 if: type == cron
 compiler: clang
-env: TARGET=linux-glibc LIBRESSL_VERSION=2.8.3
+env: TARGET=linux-glibc LIBRESSL_VERSION=2.9.2
   - os: linux
 if: type == cron
 compiler: clang
-env: TARGET=linux-glibc LIBRESSL_VERSION=2.7.5 EXTRA_OBJS="contrib/prometheus-exporter/service-prometheus.o"
+env: TARGET=linux-glibc LIBRESSL_VERSION=2.8.3 EXTRA_OBJS="contrib/prometheus-exporter/service-prometheus.o"
   - os: linux
 if: type == cron
 compiler: clang
@@ -74,13 +74,13 @@ matrix:
   - os: osx
 if: type != cron
 compiler: clang
-env: TARGET=osx FLAGS="USE_OPENSSL=1" OPENSSL_VERSION=1.1.1c
-  - os: windows
-if: type == cron
-install:
-  - choco install bash make libssl-devel cygwin-devel gcc-core libgcc1 binutils lua-devel libpcre-devel zlib-devel --source cygwin
-script:
-  - C:\\tools\\cygwin\\bin\\bash -lc 'cd $OLDPWD && make TARGE

cygwin builds

[Илья Шипицин]

hi,

I spent few days getting travis-ci / cygwin work.
no luck

I'm going to disable them unless someone knows how to resolve that


thanks,
Ilya Shipitsin

Re: Possible optimization to 51d in multithread

[Илья Шипицин]

thank you, Ben.

there's one more "finding", I guess we just mark it as intentional (inside
coverity itself), right ?

693#ifdef FIFTYONEDEGREES_H_PATTERN_INCLUDED

CID 1403837 (#1 of 1): Calling risky function (DC.WEAK_CRYPTO)dont_call:
random should not be used for security-related applications, because linear
congruential algorithms are too easy to break.

Use a compliant random number generator, such as /dev/random or /dev/urandom
on Unix-like systems, and CNG (Cryptography API: Next Generation) on
Windows.
694_51d_lru_seed = random();
695if (global_51degrees.cache_size) {
696_51d_lru_tree = lru64_new(global_51degrees.cache_size);
697}
698#endif

чт, 24 окт. 2019 г. в 13:50, Ben Shillito :

> Hi,
>
>
>
> This is intentional, as the calling method would return early if the htx
> is null (this is assuming that smp_prefetch_htx will not return a different
> result the second time?):
>
>htx = smp_prefetch_htx(smp, chn, 1);
>
>if (!htx)
>
>   return 0;
>
> Rather than doing the null check again, I will instead change the method
> to take the htx which has already been checked. This way coverity should
> not have an issue with it, we avoid having an extra null check, and we
> remove the second call to smp_prefetch_htx which was unnecessary.
>
>
>
> I will submit this as a separate patch to the other change.
>
>
>
> Thanks,
>
>
>
> Ben Shillito
> Developer
>
> [image: 51Degrees] <https://51degrees.com/>
>
> O: +44 1183 287152 
> E: b...@51degrees.com 
>  @51Degrees <http://twitter.com/51Degreesmobi>
>  51Degrees <https://www.linkedin.com/company/2171864>
>
> [image: Find out More] <https://51degrees.com/emailsig.aspx>
>
>
>
> *From:* Илья Шипицин [mailto:chipits...@gmail.com]
> *Sent:* 24 October 2019 08:04
> *To:* Ben Shillito 
> *Cc:* Willy Tarreau ; HAProxy 
> *Subject:* Re: Possible optimization to 51d in multithread
>
>
>
> Hello, Ben.
>
>
>
> since you are going to touch 51d code, can you please review the following
> coverity finding (it thinks there might be null pointer dereference) ?
>
>
>
> 239
> // No need to null check as this has already been carried out in the
>
> 240// calling method
>
>
>
> 2. returned_null: smp_prefetch_htx returns NULL (checked 35 out of 36
> times). [show details
> <https://scan6.coverity.com/eventId=28590715-1=28590715-0=94351345=%2Fsrc%2Fhttp_fetch.c=172=294>
> ]
>
>
>
> 3. var_assigned: Assigning: htx = NULL return value from smp_prefetch_htx.
>
> 241htx = smp_prefetch_htx(smp, chn, 1);
>
> 242
>
>
>
> 4. Condition i < global_51degrees.header_count, taking true branch.
>
> 243for (i = 0; i < global_51degrees.header_count; i++) {
>
> 244name.ptr = (global_51degrees.header_names + i)->area;
>
> 245name.len = (global_51degrees.header_names + i)->data;
>
> 246ctx.blk = NULL;
>
> 247
>
>
>
> CID 1403847 (#1 of 1): Dereference null return value (NULL_RETURNS)5.
> dereference: Dereferencing a pointer that might be NULL htx when calling
> http_find_header. [show details
> <https://scan6.coverity.com/eventId=28590715-16=28590715-1=94351414=%2Fsrc%2Fhttp_htx.c=54=129>
> ]
>
> 248if (http_find_header(htx, name, , 1)) {
>
> 249ws->importantHeaders[ws->importantHeadersCount
> ].header = ws->dataSet->httpHeaders + i;
>
> 250ws->importantHeaders[ws->importantHeadersCount
> ].headerValue = ctx.value.ptr;
>
> 251ws->importantHeaders[ws->importantHeadersCount
> ].headerValueLength = ctx.value.len;
>
> 252ws->importantHeadersCount++;
>
> 253}
>
> 254}
>
> 255}
>
> 256#endif
>
>
>
> ср, 23 окт. 2019 г. в 14:40, Ben Shillito :
>
> Hi Willy,
>
> Thanks for letting me know. I will get the work for this scheduled in and
> get a patch over to you.
>
> I agree it looks like a fairly small change for what I'm sure is a
> considerable performance increase for large systems.
>
> Good to see this level of attention to detail when it comes to the
> performance of HAProxy.
>
> Thanks,
>
> Ben Shillito
> Developer
> O: +44 1183 287152
> E: b...@51degrees.com
> T: @51Degrees
>
> -Original Message-
> From: Willy Tarreau [mailto:w...@1wt.eu]
> Sent: 23 October 2019 07:04
> To: Ben Shillito 
> Cc: HAProxy 
> Subject: Possible optimization to 51d in multithread
>
> Hi Ben,
>
> after 

Re: Possible optimization to 51d in multithread

[Илья Шипицин]

Hello, Ben.

since you are going to touch 51d code, can you please review the following
coverity finding (it thinks there might be null pointer dereference) ?

239
// No need to null check as this has already been carried out in the
240// calling method

2. returned_null: smp_prefetch_htx returns NULL (checked 35 out of 36
times). [show details

]

3. var_assigned: Assigning: htx = NULL return value from smp_prefetch_htx.
241htx = smp_prefetch_htx(smp, chn, 1);
242

4. Condition i < global_51degrees.header_count, taking true branch.
243for (i = 0; i < global_51degrees.header_count; i++) {
244name.ptr = (global_51degrees.header_names + i)->area;
245name.len = (global_51degrees.header_names + i)->data;
246ctx.blk = NULL;
247

CID 1403847 (#1 of 1): Dereference null return value (NULL_RETURNS)5.
dereference: Dereferencing a pointer that might be NULL htx when calling
http_find_header. [show details

]
248if (http_find_header(htx, name, , 1)) {
249ws->importantHeaders[ws->importantHeadersCount].
header = ws->dataSet->httpHeaders + i;
250ws->importantHeaders[ws->importantHeadersCount].
headerValue = ctx.value.ptr;
251ws->importantHeaders[ws->importantHeadersCount].
headerValueLength = ctx.value.len;
252ws->importantHeadersCount++;
253}
254}
255}
256#endif

ср, 23 окт. 2019 г. в 14:40, Ben Shillito :

> Hi Willy,
>
> Thanks for letting me know. I will get the work for this scheduled in and
> get a patch over to you.
>
> I agree it looks like a fairly small change for what I'm sure is a
> considerable performance increase for large systems.
>
> Good to see this level of attention to detail when it comes to the
> performance of HAProxy.
>
> Thanks,
>
> Ben Shillito
> Developer
> O: +44 1183 287152
> E: b...@51degrees.com
> T: @51Degrees
>
> -Original Message-
> From: Willy Tarreau [mailto:w...@1wt.eu]
> Sent: 23 October 2019 07:04
> To: Ben Shillito 
> Cc: HAProxy 
> Subject: Possible optimization to 51d in multithread
>
> Hi Ben,
>
> after Brian reported the thread performance regression affecting the
> pattern matchings in haproxy when relying on the LRU cache, I had a look at
> other users of the LRU cache and found that 51d.c is using it with a lock
> as well and may also suffer from a lack of linearity with threads.
>
> You may want to have a look at this patch I just committed to make the
> pattern LRU cache per thread :
>
>   403bfbb130 ("BUG/MEDIUM: pattern: make the pattern LRU cache
> thread-local and lockless")
>
> It's quite straightforward, and if you want to do the same on 51d.c, you
> just have to make your lru_tree pointer thread_local and allocate it for
> each thread in a small callback registered to be called after threads are
> initialized. Same for the call to lru64_destroy(). Then you can remove the
> lru lock and gain a lot of scalability.
>
> I'll let you have a look because I'd rather not break somehing non-
> obvious in your code and also because you know better than me how to
> benchmark the changes using the real lib and database, but if you need some
> help, just let me know.
>
> Given that it's quite small and simple a change, I'm fine with merging
> such a patch for 2.1 even a bit late (but only this, no other changes that
> are not bugfixes please).
>
> Willy
> This email and any attachments are confidential and may also be
> privileged. If you are not the named recipient, please notify the sender
> immediately and do not disclose, use, store or copy the information
> contained herein. This is an email from 51Degrees.mobi Limited, 5 Charlotte
> Close, Reading. RG47BY. T: +44 118 328 7152; E: i...@51degrees.com;
> 51Degrees.mobi Limited t/as 51Degrees.
>
>

freebsd builds are broken for few days

[Илья Шипицин]

https://cirrus-ci.com/github/haproxy/haproxy

I'll bisect if noone else knows what's going on

"official" rpm repo ?

[Илья Шипицин]

hi,

I played a bit with Fedora COPR (it is nice)


https://copr.fedorainfracloud.org/coprs/chipitsine/haproxy/build/1041926/

few questions

1) do we insist on Lua support ? (it is hard to get Lua-5.3 for EPEL7)
2) any best practice on building new packages ?

anyway, I do not insist in running my very own repo (I started to work with
packages because I did not find any good repo with recent haproxy builds),
I can give those packages back to haproxy itself (if there's an interest)

thanks,
Ilya Shipitsin

using hashicorp vault for storing SSL certs

[Илья Шипицин]

hello,

is anybody using https://www.vaultproject.io/docs/secrets/pki/index.html
for storing certs ? (I want to avoid reinventing the wheel here)

thanks,
Ilya Shipitcin

Re: [PATCH] improving github experience, kindly ask people to reproduce bugs on latest haproxy

[Илья Шипицин]

пн, 23 сент. 2019 г. в 22:02, Willy Tarreau :

> Hi guys,
>
> On Fri, Sep 20, 2019 at 03:47:17PM +0200, Lukas Tribus wrote:
> > Hello Patrick,
> >
> >
> > > I dunno, I've personally never been fond of it when bug reporters are
> blindly asked to upgrade to the latest version.
> >
> > Everything you are saying is relevant for a support environment, like
> > the mailing list or discourse. However the bug tracker is not a
> > support forum, it's a bug tracker only. We need factual data,
> > everything else belongs to the support channels. We needed this to
> > file known bugs and features requests so we stop forgetting about them
> > in the stream of emails and to coordinate fixes of known, confirmed
> > bugs.
> >
> > Nobody is saying that problems need to be reported on Github. People
> > seeking help ought to report their issues to the mailing list. If on
> > the other hand those people are willing to file a bug report - and
> > this implies some groundwork for them - then that's great, because it
> > helps us.
> >
> > However it needs to be clear that the issue tracker in Github is not a
> > support forum, and that filing a report will needs some ground work.
> > It is also *always* a good idea to discuss an issue on the ML first,
> > before filing an issue.
>
> I'm a bit torn between you two on this actually. There are several
> reasons. The first one is that nobody is running the latest version
> since a latest exists after a commit is pushed, and that at this game
> it can circle forever. Some would say "but not released doesn't count"
> and I'd argue that *new* bugs affecting -dev are even more important
> to me than older ones since they are regressions, which is also why we
> now have the CI running.
>
> However I also agree that it's important to encourage users to upgrade
> *when they can*. Patrick is right regarding the difficulty to upgrade
> in some environments, I've worked in such environments and sometimes
> you get a very strange behavior, you collect traces as fast as you can
> and you want to archive your report in a safe place, shared by others,
> knowing that it's incomplete but that it may help in the future when
> it's confronted to another apparently similar one, coming from someone
> with much less traces. In this regard the issue tracker is convenient
> to keep some warm issues available and not forgotten even if we consider
> them incomplete but credible.
>
> Similarly some issues which trigger very late can by definition never
> happen on the latest version. For example, just remove 3 lines in the
> scheduler to allow to loop at the end of the tree back to the beginning
> and suddenly all haproxy code deployed will fail after 49.7 days. And
> we do want to get such reports that are extremely rare and valuable by
> nature.
>
> My hope is that we can encourage good faith among the reporters. I'd
> suggest changing the question to something like this instead :
>
>   [ ] I confirm that I did my best to upgrade to the latest version
>   before reporting this bug and I am also conscious that developers
>   tend to be much less reactive on older versions.
>

I'm ok with that.


>
> It doesn't necessarily *have* to be a check box, it just needs to be
> prominent so that it's well understood.
>
> With all this said, we're starting to see more bugs with multiple
> participants and this is encouraging because that's exactly what is
> needed to help narrow down an issue. Thus I'm not for really filtering
> at the door, but rather making sure that the reporter thinks twice and
> decides.
>
> Typically those who install their fresh new PC with a default distro,
> start the default haproxy (without updating the distro) and report an
> issue should be reminded about trying again and upgrading first. But
> those who have been chasing a bug for a month (like Veiko in the other
> thread) an finally been hit by it with subtantial information (not
> necessarily traces/conf, sometimes context explanations) should be
> able to post if they think it does provide some value.
>
> > > In corporate environments, it can be difficult to perform such
> upgrades.
> > > Sometimes these issues are only reproducible in production
> environments.
> >
> > I know that. But that's not a bug report, it's a support request
>
> Not necessarily. Regarding the example about the time looping at 49.7 days,
> I actually experienced it with other products long ago. It definitely is a
> bug. Just like I've seen some equipements get their VRRP desynchronized
> progressively over time, the amount of desync grew by a few seconds in sine
> waves of something like 24 days. The initially observable issue was that
> sometimes there would be a quick failover and a switch back, and that over
> a long period that would happen more often then less often. It's quite
> difficult to qualify such a think but it definitely is a bug an not someone
> asking for help. Sharing elements as they come can be useful, provided the
> person 

[PATCH] improving github experience, kindly ask people to reproduce bugs on latest haproxy

[Илья Шипицин]

hello,

please find attached patch

Ilya Shipitsin
From 2d4b8bf119bf470c2c607a1fd22cee8267e6843b Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Fri, 20 Sep 2019 00:07:11 +0500
Subject: [PATCH] DOC: kindly ask people whether they run latest HAProxy

---
 .github/ISSUE_TEMPLATE/Bug.md | 4 
 1 file changed, 4 insertions(+)

diff --git a/.github/ISSUE_TEMPLATE/Bug.md b/.github/ISSUE_TEMPLATE/Bug.md
index 54f0475f7..5c7eb78b5 100644
--- a/.github/ISSUE_TEMPLATE/Bug.md
+++ b/.github/ISSUE_TEMPLATE/Bug.md
@@ -4,6 +4,10 @@ about: Report a problem with HAProxy to help us resolve it.
 labels: 'type: bug, status: needs-triage'
 ---
 
+### Prerequisites
+
+* [ ] Are you running the latest version of HAProxy ?
+
 

[PATCH] enabling slz, pcre2 builds in travis-ci

[Илья Шипицин]

hello,

please consider attached patch

Ilya Shipitsin
From b65dc30f1c07aca067ebb31d373833fe66486b7b Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Thu, 19 Sep 2019 23:32:30 +0500
Subject: [PATCH] BUILD: travis-ci: add PCRE2, SLZ build

---
 .travis.yml | 9 -
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/.travis.yml b/.travis.yml
index 8fb906409..879a676ae 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -13,7 +13,7 @@ env:
 
 addons:
   apt:
-packages: [ liblua5.3-dev, libsystemd-dev ]
+packages: [ liblua5.3-dev, libsystemd-dev, libpcre2-dev ]
 
 cache:
   directories:
@@ -74,6 +74,13 @@ matrix:
   - choco install bash make libssl-devel cygwin-devel gcc-core libgcc1 binutils lua-devel libpcre-devel zlib-devel --source cygwin
 script:
   - C:\\tools\\cygwin\\bin\\bash -lc 'cd $OLDPWD && make TARGET=cygwin USE_ZLIB=1 USE_PCRE=1 USE_PCRE_JIT=1 USE_LUA=1 USE_OPENSSL=1 USE_THREAD=1 && ./haproxy -vv'
+  - os: linux
+if: type == cron
+compiler: clang
+env: TARGET=linux-glibc FLAGS="USE_SLZ=1 USE_PCRE2=1 USE_PCRE2_JIT=1 USE_LUA=1 USE_OPENSSL=1 USE_SYSTEMD=1 USE_WURFL=1 WURFL_INC=contrib/wurfl WURFL_LIB=contrib/wurfl USE_51DEGREES=1"
+before_script:
+  - git clone http://git.1wt.eu/git/libslz.git/
+  - cd libslz && make && make PREFIX=${HOME}/opt install && cd ..
 
 install:
   - git clone https://github.com/VTest/VTest.git ../vtest
-- 
2.20.1

Re: PATCH: install golang-1.13 during travis-ci build as it is required for BoringSSL

[Илья Шипицин]

Willy, can you apply this ?

boringssl builds are bloody murder

пн, 16 сент. 2019 г. в 16:14, Илья Шипицин :

> please see attached patch
>

PATCH: install golang-1.13 during travis-ci build as it is required for BoringSSL

[Илья Шипицин]

please see attached patch
From 5fa12aec93f1e8989e06628fc5e41fc2556f532b Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Mon, 16 Sep 2019 16:13:10 +0500
Subject: [PATCH] BUILD: CI: install golang-1.13 when building BoringSSL

---
 scripts/build-ssl.sh | 4 
 1 file changed, 4 insertions(+)

diff --git a/scripts/build-ssl.sh b/scripts/build-ssl.sh
index cec81e04a..384438a22 100755
--- a/scripts/build-ssl.sh
+++ b/scripts/build-ssl.sh
@@ -79,6 +79,10 @@ fi
 
 if [ ! -z ${BORINGSSL+x} ]; then
 	(
+
+	# travis-ci comes with go-1.11, while boringssl requires go-1.13
+	eval "$(curl -sL https://raw.githubusercontent.com/travis-ci/gimme/master/gimme | GIMME_GO_VERSION=1.13 bash)"
+
 download_boringssl
 	cd download-cache/boringssl
 if [ -d build ]; then rm -rf build; fi
-- 
2.20.1

Re: Linux Builds broken on Travis CI

[Илья Шипицин]

it turned out that ASAN is "the root cause" of those failures. let us
disable it for a while (I attached patch)

пт, 13 сент. 2019 г. в 19:23, Илья Шипицин :

>
>
> On Fri, Sep 13, 2019, 3:49 PM Willy Tarreau  wrote:
>
>> On Fri, Sep 13, 2019 at 03:45:21PM +0500,  ??? wrote:
>> > now build fails with
>> >
>> > "** h1 debug|[ALERT] 255/081449 (8721) : failed to allocate resources
>> for
>> > thread 1."
>>
>> That's exactly the issues I was talking about that started to happen
>> at an increasing frequency over the last few weeks.
>>
>> > no more failures due to leaks.
>>
>> Great! What do you think about leaving the tests only for the cron tasks ?
>>
>
> Give me few days ))
>
>
>
>> Willy
>>
>
From 8254d7dbd00cf9c6f06aa434d1bf156a6b04d7e0 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Sat, 14 Sep 2019 21:18:49 +0500
Subject: [PATCH] BUILD: CI: temporarily disable ASAN

it turned out that ASAN breaks things. until this is resolved,
let us disable ASAN
---
 .travis.yml | 1 -
 1 file changed, 1 deletion(-)

diff --git a/.travis.yml b/.travis.yml
index a65662496..8fb906409 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -82,7 +82,6 @@ install:
   - scripts/build-ssl.sh > build-ssl.log 2>&1 || (cat build-ssl.log && exit 1)
 
 script:
-  - if [ "${CC}"  = "clang" ]; then export FLAGS="$FLAGS USE_OBSOLETE_LINKER=1" DEBUG_CFLAGS="-g -fsanitize=address" LDFLAGS="-fsanitize=address"; fi
   - make -C contrib/wurfl
   - make -j3 CC=$CC V=1 TARGET=$TARGET $FLAGS DEBUG_CFLAGS="$DEBUG_CFLAGS" LDFLAGS="$LDFLAGS" 51DEGREES_SRC="$FIFTYONEDEGREES_SRC" EXTRA_OBJS="$EXTRA_OBJS"
   - if [ "${TRAVIS_OS_NAME}" = "linux" ]; then export LD_LIBRARY_PATH="${HOME}/opt/lib:${LD_LIBRARY_PATH:-}"; fi
-- 
2.20.1

Re: Linux Builds broken on Travis CI

[Илья Шипицин]

On Fri, Sep 13, 2019, 3:49 PM Willy Tarreau  wrote:

> On Fri, Sep 13, 2019 at 03:45:21PM +0500,  ??? wrote:
> > now build fails with
> >
> > "** h1 debug|[ALERT] 255/081449 (8721) : failed to allocate resources for
> > thread 1."
>
> That's exactly the issues I was talking about that started to happen
> at an increasing frequency over the last few weeks.
>
> > no more failures due to leaks.
>
> Great! What do you think about leaving the tests only for the cron tasks ?
>

Give me few days ))



> Willy
>

Re: Linux Builds broken on Travis CI

[Илья Шипицин]

now build fails with

"** h1 debug|[ALERT] 255/081449 (8721) : failed to allocate resources for
thread 1."


no more failures due to leaks.



пт, 13 сент. 2019 г. в 13:33, Willy Tarreau :

> On Fri, Sep 13, 2019 at 01:23:12PM +0500,  ??? wrote:
> > Build was failed due to memory leak detected by asan
> >
> > https://github.com/haproxy/haproxy/issues/256
> >
> >
> > I think we can change the way asan works, I.e. log errors and do not stop
> > tests
>
> I didn't even notice it was this one because we've had too many errors
> reported over the last weeks as I mentioned. At this point I'd rather
> do the opposite and possibly keep asan (if it reports valid things only)
> and drop the tests which randomly fail 50% of the time on this
> infrastructure.
>

I meant "collect and report leaks separately from other failures".
I'll send patch soon


>
> Willy
>

Re: Linux Builds broken on Travis CI

[Илья Шипицин]

Build was failed due to memory leak detected by asan

https://github.com/haproxy/haproxy/issues/256


I think we can change the way asan works, I.e. log errors and do not stop
tests



On Fri, Sep 13, 2019, 7:59 AM Willy Tarreau  wrote:

> Hi Tim,
>
> On Fri, Sep 06, 2019 at 04:30:24PM +0200, Tim Düsterhus wrote:
> > Dear List
> >
> > something between 02bac85bee664976f6dcecc424864e9fb99975be and
> > f909c91e8a739b9ef7409b399259201fe883771c broke all the Linux builds on
> > Travis CI:
> >
> > - 41 reg tests fail with a timeout
> > - 3 reg tests pass
> >
> > FreeBSD works fine.
> >
> > Somebody really ought to take a look. I might try to bisect if I find a
> > bit of spare time. If someone beats me to it: Go ahead.
>
> I've been quite annoyed with this a number of times and ended up not
> looking at build reports anymore due to this. I've spent some time
> looking at the cause as well and bisecting, coming to the conclusion
> that apparently the travis VMs are regularly overloaded. Most of the
> times we see TCP connection timeouts on the loop back preventing the
> vtest client from reaching haproxy! I've even seen a number of "out
> of memory" messages hitting the client. It's possible that their
> hypervisor is sometimes running out of memory. Maybe their service
> is abused by other projects which induce a huge load. At some point I
> used to click "build again", which managed to randomly work, but I
> gave up, being used to seeing this constantly red :-(
>
> I think that we should simply disable reg tests and stick to build
> tests only. There's nothing worse than getting used to seeing errors,
> as by not seeing a difference between a build error and a test error
> we get trained to ignore results.
>
> Maybe we can keep the reg tests for cron jobs, but given that they
> similarly fail I don't see the benefit either.
>
> I too would like to see them turn green again :-/
>
> Cheers,
> Willy
>
>

Re: Docker Image update to OpenSSL 1.1.1d

[Илья Шипицин]

чт, 12 сент. 2019 г. в 01:01, Aleksandar Lazic :

> Hi.
>
> Am 11.09.2019 um 20:48 schrieb Илья Шипицин:
> > Hello,
> >
> > it was a surprize for me that official images are also available (I used
> to
> > think your images are only available)
> >
> > https://hub.docker.com/_/haproxy
> >
> > is there some documentation when your images should be used (instead of
> official) ?
>
> Well in the past there was no images with TLS 1.3 and LUA enabled and
> therefore
> I build my own, looks like this is nowadays not more the case so I think
> the
> official images are quite good.
>
> Thanks for the hint.
>
> The biggest difference between the official Images and my is that you can
> run my
> images also on OpenShift without adaption as I don't use any user or group
> change to a specific user which makes a a lot of troubles on OpenShift and
> the
> default binds are not on privileged ports.
>

it's interesting. it might worth to be ported to official images.


>
> The source for my image is on gitlab, that's also one reason why there is
> not a
> automatic build on docker hub as docker hub does not support gitlab for
> automatic builds.
>
> https://gitlab.com/aleks001/haproxy20-centos/blob/master/Dockerfile
>
> Best regards
> Aleks
>
> > ср, 11 сент. 2019 г. в 22:58, Aleksandar Lazic  > <mailto:al-hapr...@none.at>>:
> >
> > Hi.
> >
> > I have updated the image to the latest OpenSSL version 1.1.1d
> >
> > https://hub.docker.com/r/me2digital/haproxy20-centos
> >
> > ```
> > $ docker run --rm --entrypoint /usr/local/sbin/haproxy
> > [MASKED]/haproxy20-centos -vv
> > HA-Proxy version 2.0.5 2019/08/16 - https://haproxy.org/
> > Build options :
> >   TARGET  = linux-glibc
> >   CPU = generic
> >   CC  = gcc
> >   CFLAGS  = -O2 -g -fno-strict-aliasing
> -Wdeclaration-after-statement -fwrapv
> > -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter
> > -Wno-old-style-declaration -Wno-ignored-qualifiers -Wno-clobbered
> > -Wno-missing-field-initializers -Wtype-limits
> >   OPTIONS = USE_PCRE=1 USE_PCRE_JIT=1 USE_PTHREAD_PSHARED=1
> USE_REGPARM=1
> > USE_OPENSSL=1 USE_LUA=1 USE_SLZ=1
> >
> > Feature list : +EPOLL -KQUEUE -MY_EPOLL -MY_SPLICE +NETFILTER +PCRE
> +PCRE_JIT
> > -PCRE2 -PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD +PTHREAD_PSHARED
> +REGPARM
> > -STATIC_PCRE -STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE
> +LIBCRYPT
> > +CRYPT_H -VSYSCALL +GETADDRINFO +OPENSSL +LUA +FUTEX +ACCEPT4
> -MY_ACCEPT4 -ZLIB
> > +SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL
> -SYSTEMD
> > -OBSOLETE_LINKER +PRCTL +THREAD_DUMP -EVPORTS
> >
> > Default settings :
> >   bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
> >
> > Built with multi-threading support (MAX_THREADS=64, default=1).
> > Built with OpenSSL version : OpenSSL 1.1.1d  10 Sep 2019
> > Running on OpenSSL version : OpenSSL 1.1.1d  10 Sep 2019
> > OpenSSL library supports TLS extensions : yes
> > OpenSSL library supports SNI : yes
> > OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
> > Built with Lua version : Lua 5.3.5
> > Built with network namespace support.
> > Built with transparent proxy support using: IP_TRANSPARENT
> IPV6_TRANSPARENT
> > IP_FREEBIND
> > Built with libslz for stateless compression.
> > Compression algorithms supported : identity("identity"),
> deflate("deflate"),
> > raw-deflate("deflate"), gzip("gzip")
> > Built with PCRE version : 8.32 2012-11-30
> > Running on PCRE version : 8.32 2012-11-30
> > PCRE library supports JIT : yes
> > Encrypted password support via crypt(3): yes
> > Built with the Prometheus exporter as a service
> >
> > Available polling systems :
> >   epoll : pref=300,  test result OK
> >poll : pref=200,  test result OK
> >  select : pref=150,  test result OK
> > Total: 3 (3 usable), will use epoll.
> >
> > Available multiplexer protocols :
> > (protocols marked as  cannot be specified using 'proto'
> keyword)
> >   h2 : mode=HTXside=FE|BE mux=H2
> >   h2 : mode=HTTP   side=FEmux=H2
> > : mode=HTXside=FE|BE mux=H1
> > : mode=TCP|HTTP   side=FE|BE mux=PASS
> >
> > Available services :
> > prometheus-exporter
> >
> > Available filters :
> > [SPOE] spoe
> > [COMP] compression
> > [CACHE] cache
> > [TRACE] trace
> > ```
> >
> > Regards
> > Aleks
> >
>
>

Re: Docker Image update to OpenSSL 1.1.1d

[Илья Шипицин]

oops.


I did have a look at https://hub.docker.com/search?q=haproxytech=image
(built from https://github.com/haproxytech )


so... there are many many many docker images.
when should I use either of these images?

ср, 11 сент. 2019 г. в 23:48, Илья Шипицин :

> Hello,
>
> it was a surprize for me that official images are also available (I used
> to think your images are only available)
>
> https://hub.docker.com/_/haproxy
>
> is there some documentation when your images should be used (instead of
> official) ?
>
> ср, 11 сент. 2019 г. в 22:58, Aleksandar Lazic :
>
>> Hi.
>>
>> I have updated the image to the latest OpenSSL version 1.1.1d
>>
>> https://hub.docker.com/r/me2digital/haproxy20-centos
>>
>> ```
>> $ docker run --rm --entrypoint /usr/local/sbin/haproxy
>> [MASKED]/haproxy20-centos -vv
>> HA-Proxy version 2.0.5 2019/08/16 - https://haproxy.org/
>> Build options :
>>   TARGET  = linux-glibc
>>   CPU = generic
>>   CC  = gcc
>>   CFLAGS  = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement
>> -fwrapv
>> -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter
>> -Wno-old-style-declaration -Wno-ignored-qualifiers -Wno-clobbered
>> -Wno-missing-field-initializers -Wtype-limits
>>   OPTIONS = USE_PCRE=1 USE_PCRE_JIT=1 USE_PTHREAD_PSHARED=1 USE_REGPARM=1
>> USE_OPENSSL=1 USE_LUA=1 USE_SLZ=1
>>
>> Feature list : +EPOLL -KQUEUE -MY_EPOLL -MY_SPLICE +NETFILTER +PCRE
>> +PCRE_JIT
>> -PCRE2 -PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD +PTHREAD_PSHARED +REGPARM
>> -STATIC_PCRE -STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT
>> +CRYPT_H -VSYSCALL +GETADDRINFO +OPENSSL +LUA +FUTEX +ACCEPT4 -MY_ACCEPT4
>> -ZLIB
>> +SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL
>> -SYSTEMD
>> -OBSOLETE_LINKER +PRCTL +THREAD_DUMP -EVPORTS
>>
>> Default settings :
>>   bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
>>
>> Built with multi-threading support (MAX_THREADS=64, default=1).
>> Built with OpenSSL version : OpenSSL 1.1.1d  10 Sep 2019
>> Running on OpenSSL version : OpenSSL 1.1.1d  10 Sep 2019
>> OpenSSL library supports TLS extensions : yes
>> OpenSSL library supports SNI : yes
>> OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
>> Built with Lua version : Lua 5.3.5
>> Built with network namespace support.
>> Built with transparent proxy support using: IP_TRANSPARENT
>> IPV6_TRANSPARENT
>> IP_FREEBIND
>> Built with libslz for stateless compression.
>> Compression algorithms supported : identity("identity"),
>> deflate("deflate"),
>> raw-deflate("deflate"), gzip("gzip")
>> Built with PCRE version : 8.32 2012-11-30
>> Running on PCRE version : 8.32 2012-11-30
>> PCRE library supports JIT : yes
>> Encrypted password support via crypt(3): yes
>> Built with the Prometheus exporter as a service
>>
>> Available polling systems :
>>   epoll : pref=300,  test result OK
>>poll : pref=200,  test result OK
>>  select : pref=150,  test result OK
>> Total: 3 (3 usable), will use epoll.
>>
>> Available multiplexer protocols :
>> (protocols marked as  cannot be specified using 'proto' keyword)
>>   h2 : mode=HTXside=FE|BE mux=H2
>>   h2 : mode=HTTP   side=FEmux=H2
>> : mode=HTXside=FE|BE mux=H1
>> : mode=TCP|HTTP   side=FE|BE mux=PASS
>>
>> Available services :
>> prometheus-exporter
>>
>> Available filters :
>> [SPOE] spoe
>> [COMP] compression
>> [CACHE] cache
>> [TRACE] trace
>> ```
>>
>> Regards
>> Aleks
>>
>>

Re: Docker Image update to OpenSSL 1.1.1d

[Илья Шипицин]

Hello,

it was a surprize for me that official images are also available (I used to
think your images are only available)

https://hub.docker.com/_/haproxy

is there some documentation when your images should be used (instead of
official) ?

ср, 11 сент. 2019 г. в 22:58, Aleksandar Lazic :

> Hi.
>
> I have updated the image to the latest OpenSSL version 1.1.1d
>
> https://hub.docker.com/r/me2digital/haproxy20-centos
>
> ```
> $ docker run --rm --entrypoint /usr/local/sbin/haproxy
> [MASKED]/haproxy20-centos -vv
> HA-Proxy version 2.0.5 2019/08/16 - https://haproxy.org/
> Build options :
>   TARGET  = linux-glibc
>   CPU = generic
>   CC  = gcc
>   CFLAGS  = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement
> -fwrapv
> -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter
> -Wno-old-style-declaration -Wno-ignored-qualifiers -Wno-clobbered
> -Wno-missing-field-initializers -Wtype-limits
>   OPTIONS = USE_PCRE=1 USE_PCRE_JIT=1 USE_PTHREAD_PSHARED=1 USE_REGPARM=1
> USE_OPENSSL=1 USE_LUA=1 USE_SLZ=1
>
> Feature list : +EPOLL -KQUEUE -MY_EPOLL -MY_SPLICE +NETFILTER +PCRE
> +PCRE_JIT
> -PCRE2 -PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD +PTHREAD_PSHARED +REGPARM
> -STATIC_PCRE -STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT
> +CRYPT_H -VSYSCALL +GETADDRINFO +OPENSSL +LUA +FUTEX +ACCEPT4 -MY_ACCEPT4
> -ZLIB
> +SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL -SYSTEMD
> -OBSOLETE_LINKER +PRCTL +THREAD_DUMP -EVPORTS
>
> Default settings :
>   bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
>
> Built with multi-threading support (MAX_THREADS=64, default=1).
> Built with OpenSSL version : OpenSSL 1.1.1d  10 Sep 2019
> Running on OpenSSL version : OpenSSL 1.1.1d  10 Sep 2019
> OpenSSL library supports TLS extensions : yes
> OpenSSL library supports SNI : yes
> OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
> Built with Lua version : Lua 5.3.5
> Built with network namespace support.
> Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT
> IP_FREEBIND
> Built with libslz for stateless compression.
> Compression algorithms supported : identity("identity"),
> deflate("deflate"),
> raw-deflate("deflate"), gzip("gzip")
> Built with PCRE version : 8.32 2012-11-30
> Running on PCRE version : 8.32 2012-11-30
> PCRE library supports JIT : yes
> Encrypted password support via crypt(3): yes
> Built with the Prometheus exporter as a service
>
> Available polling systems :
>   epoll : pref=300,  test result OK
>poll : pref=200,  test result OK
>  select : pref=150,  test result OK
> Total: 3 (3 usable), will use epoll.
>
> Available multiplexer protocols :
> (protocols marked as  cannot be specified using 'proto' keyword)
>   h2 : mode=HTXside=FE|BE mux=H2
>   h2 : mode=HTTP   side=FEmux=H2
> : mode=HTXside=FE|BE mux=H1
> : mode=TCP|HTTP   side=FE|BE mux=PASS
>
> Available services :
> prometheus-exporter
>
> Available filters :
> [SPOE] spoe
> [COMP] compression
> [CACHE] cache
> [TRACE] trace
> ```
>
> Regards
> Aleks
>
>

Re: Coverity scan findings

[Илья Шипицин]

it depends on how haproxy is built (number of flags)

we use most of available options when testing on coverity

https://github.com/haproxy/haproxy/blob/master/.travis.yml#L8

can you share build command ? we may also set up sonar in travis-ci
schedules.

(personally, I find sonar too much noisy, but I agree, it finds bugs
sometimes)

ср, 11 сент. 2019 г. в 13:08, GARDAIS Ionel <
ionel.gard...@tech-advantage.com>:

>
> > On Tue, Sep 10, 2019 at 08:29:38PM +0500,  ??? wrote:
> > > those findings are mostly mess (maybe, except few real bugs).
> > > I do not mind sharing those findings with community, Willy ?
> > > we need more manpower here.
> >
> > Oh no problem! I'm not the one asking to hide bugs, the more eyeballs
> > on bug reports, the faster these ones will be sorted out! Also if one
> > fears that this could help a black hat guy find a vulnerability and
> > exploit it, mind you that these people already spend time scanning the
> > same code (with and without tools) and spot bugs in advance without
> > relying on our public reports anyway.
>
>
> Please note that Sonarqube is scanning haproxy code too.
> Results are available at https://sonarcloud.io/dashboard?id=haproxy
>
> Some results are false positive but some are worth looking at.
> --
> 232 avenue Napoleon BONAPARTE 92500 RUEIL MALMAISON
> Capital EUR 219 300,00 - RCS Nanterre B 408 832 301 - TVA FR 09 408 832 301
>
>

Re: [PATCH] basic CentOS6 CI

[Илья Шипицин]

Good good good.

On Sun, Sep 8, 2019, 3:11 PM Willy Tarreau  wrote:

> On Fri, Sep 06, 2019 at 11:24:44PM +0500,  ??? wrote:
> > here's another CI patch (which skips alpn testing on CentOS 6)
>
> Thanks Ilya, I've merged it and added a commit message based on the
> explanation in the modified file.
>
> Willy
>

Re: [PATCH] basic CentOS6 CI

[Илья Шипицин]

hello,

here's another CI patch (which skips alpn testing on CentOS 6)

пт, 6 сент. 2019 г. в 15:48, Илья Шипицин :

> great.
>
> I'll resolve ALPN testing soon. let those builds be red for a while.
>
> пт, 6 сент. 2019 г. в 14:45, Willy Tarreau :
>
>> On Mon, Sep 02, 2019 at 12:10:11AM +0500,  ??? wrote:
>> > please review the following patch
>>
>> No particular opinion on it so I merged it.
>>
>> Thanks Ilya.
>> Willy
>>
>
From f106bf905cc6fb21ffc0e4f1769cef3b6d22c361 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Fri, 6 Sep 2019 23:18:14 +0500
Subject: [PATCH] BUILD: CI: skip
 reg-tests/connection/proxy_protocol_random_fail.vtc on CentOS 6

---
 .cirrus.yml | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/.cirrus.yml b/.cirrus.yml
index 095e27b0d..5d3870954 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -24,4 +24,6 @@ centos_6_task:
 - make CC=cc V=1 TARGET=linux-glibc-legacy USE_ZLIB=1 USE_PCRE=1 USE_OPENSSL=1
 - ./haproxy -vv
 - ldd haproxy
+# remove alpn reg-test (CentOS 6 does not support alpn)
+- rm reg-tests/connection/proxy_protocol_random_fail.vtc
 - env VTEST_PROGRAM=../vtest/vtest make reg-tests || (for folder in /tmp/*regtest*/vtc.*; do cat $folder/INFO $folder/LOG; done && exit 1)
-- 
2.20.1