Tim, let us move this activity to GH issue ?
as for failing osx builds, I think it might be due to long folder name https://github.com/TimWolla/haproxy/runs/924097118?check_suite_focus=true#step:10:361 it is 100+ characters. there's also unix socket in that dir, which has limitation on filename length. I use TMPDIR=/tmp in travis to make path shorter (but it would be nice to output logs on failure to see that error in log) ср, 29 июл. 2020 г. в 21:49, Tim Duesterhus <t...@bastelstu.be>: > Travis is becoming overall increasingly unreliable lately. We've already > seen that the timing sensitive tests regularly fail and thus they were > disabled. > > GitHub Actions VMs are working well, possibly allowing to use custom > runners > for special tasks in the future. > > In addition to this better performance its workflow configuration language > is more expressive compared to the Travis CI one. Specifically the build > matrix does not need to be specified in YAML. Instead it can be generated > ad-hoc using a script. This allows us to cleanly define the various build > configurations without having an unreadable 80 line mess where the flags > are inconsistently activated. As an example in the current Travis CI > configuration the prometheus exporter is tested together with LibreSSL > 2.9.2 > for whatever reason. > > In addition to all the previous points the UI of Travis is not that nice. > On GitHub you are just seeing that "Travis failed" without any details > which > exact job failed. This requires you to visit the slow Travis page and look > up the details there. GitHub Actions creates a single entry for each > configuration that is tested, allowing you to see the details without > needing > to leave GitHub. > > This new GitHub Actions workflow aims to reproduce the configurations > tested > in Travis. It comes close, but is not completely there, yet. Consider this > patch a proof of concept that will evolve in the future, ideally with > Ilya's > expertise. > > The current configurations are as follows. Each one is tested with both gcc > and clang. > - All features disabled (no USE flags) > - All features enabled (all USE flags) > - Standalone test of each of the supported compression libraries: > - USE_ZLIB=1 > - USE_SLZ=1 > - Standalone test of various SSL libraries: > - stock (the SSL on the VM) > - OpenSSL 1.0.2u, 1.1.0l, 1.1.1f > - LibreSSL 2.9.2, 3.0.2, 3.1.1 > > Future additions of new tests should take care to not test unrelated stuff. > Instead a distinct configuration should be added. > > Additionally there is a Mac OS test with clang and all features disabled. > > Known issues: > - Apparently the git commit is not properly detected during build. The HEAD > currently shows as 2.3-dev1. > - Mac OS currently fails. > - The detailed logs are not printed for failing tests. > --- > .github/workflows/vtest.yml | 175 ++++++++++++++++++++++++++++++++++++ > 1 file changed, 175 insertions(+) > create mode 100644 .github/workflows/vtest.yml > > diff --git a/.github/workflows/vtest.yml b/.github/workflows/vtest.yml > new file mode 100644 > index 000000000..d94322474 > --- /dev/null > +++ b/.github/workflows/vtest.yml > @@ -0,0 +1,175 @@ > +# Copyright 2019 Ilya Shipitsin <chipits...@gmail.com> > +# Copyright 2020 Tim Duesterhus <t...@bastelstu.be> > +# > +# This program is free software; you can redistribute it and/or > +# modify it under the terms of the GNU General Public License > +# as published by the Free Software Foundation; either version > +# 2 of the License, or (at your option) any later version. > + > +name: VTest > + > +on: > + push: > + > +jobs: > + generate-matrix: > + name: Generate Build Matrix > + runs-on: ubuntu-latest > + outputs: > + matrix: ${{ steps.set-matrix.outputs.matrix }} > + steps: > + - name: Generate Build Matrix > + id: set-matrix > + shell: python > + run: | > + import json > + > + matrix = [] > + os = "ubuntu-latest" > + TARGET = "linux-glibc" > + for CC in [ "gcc", "clang" ]: > + matrix.append({ > + "name": "{} with no features enabled on {}".format(CC, os), > + "os": os, > + "TARGET": TARGET, > + "CC": CC, > + "FLAGS": "" > + }) > + > + matrix.append({ > + "name": "{} with all features enabled on {}".format(CC, os), > + "os": os, > + "TARGET": TARGET, > + "CC": CC, > + "FLAGS": " ".join([ > + "USE_ZLIB=1", > + "USE_PCRE=1", > + "USE_PCRE_JIT=1", > + "USE_LUA=1", > + "USE_OPENSSL=1", > + "USE_SYSTEMD=1", > + # "USE_WURFL=1", > + # "WURFL_INC=contrib/wurfl", > + # "WURFL_LIB=contrib/wurfl", > + # "USE_DEVICEATLAS=1", > + # "DEVICEATLAS_SRC=contrib/deviceatlas", > + # "USE_51DEGREES=1", > + # "FIFTYONEDEGREES_SRC=contrib/51d/src/pattern", > + ]) > + }) > + > + for compression in [ > + "USE_SLZ=1", > + "USE_ZLIB=1" > + ]: > + matrix.append({ > + "name": "{} with {} on {}".format(CC, compression, os), > + "os": os, > + "TARGET": TARGET, > + "CC": CC, > + "compression": compression, > + "FLAGS": compression > + }) > + > + for ssl in [ > + "stock", > + "OPENSSL_VERSION=1.0.2u", > + "OPENSSL_VERSION=1.1.0l", > + "OPENSSL_VERSION=1.1.1f", > + "LIBRESSL_VERSION=2.9.2", > + "LIBRESSL_VERSION=3.0.2", > + "LIBRESSL_VERSION=3.1.1", > + ]: > + flags = ["USE_OPENSSL=1"] > + if ssl != "stock": > + flags.append("SSL_LIB=${HOME}/opt/lib") > + flags.append("SSL_INC=${HOME}/opt/include") > + matrix.append({ > + "name": "{} with {} SSL on {}".format(CC, ssl, os), > + "os": os, > + "TARGET": TARGET, > + "CC": CC, > + "ssl": ssl, > + "FLAGS": " ".join(flags) > + }) > + > + os = "macos-latest" > + TARGET = "osx" > + for CC in [ "clang" ]: > + matrix.append({ > + "name": "{} with no features enabled on {}".format(CC, os), > + "os": os, > + "TARGET": TARGET, > + "CC": CC, > + "FLAGS": "" > + }) > + > + print("::set-output name=matrix::{}".format(json.dumps({ > + "include": matrix > + }))) > + Test: > + name: ${{ matrix.name }} > + needs: generate-matrix > + runs-on: ${{ matrix.os }} > + strategy: > + matrix: ${{ fromJson(needs.generate-matrix.outputs.matrix) }} > + fail-fast: false > + steps: > + - uses: actions/checkout@v2 > + with: > + fetch-depth: 100 > + - name: Install apt dependencies. > + if: ${{ startsWith(matrix.os, 'ubuntu-') }} > + run: | > + sudo apt-get install -y \ > + liblua5.3-dev \ > + libpcre2-dev \ > + libsystemd-dev \ > + ninja-build \ > + socat > + - name: Install brew dependencies. > + if: ${{ startsWith(matrix.os, 'macos-') }} > + run: | > + brew install socat > + brew install lua > + - name: Install VTest. > + run: | > + curl -fsSL https://github.com/vtest/VTest/archive/master.tar.gz > -o VTest.tar.gz > + mkdir VTest > + tar xvf VTest.tar.gz -C VTest --strip-components=1 > + make -C VTest -j$(nproc) FLAGS="-O2 -s -Wall" > + sudo install -m755 VTest/vtest /usr/local/bin/vtest > + - name: Install SLZ > + if: ${{ matrix.compression == 'USE_SLZ=1' }} > + run: | > + curl -fsSL > https://github.com/wtarreau/libslz/archive/master.tar.gz -o libslz.tar.gz > + mkdir libslz > + tar xvf libslz.tar.gz -C libslz --strip-components=1 > + make -C libslz > + sudo make -C libslz install > + - name: Install SSL ${{ matrix.ssl }} > + if: ${{ matrix.ssl && matrix.ssl != 'stock' }} > + run: env ${{ matrix.ssl }} scripts/build-ssl.sh > + - name: Compile HAProxy with ${{ matrix.CC }}. > + run: | > + make -j$(nproc) all \ > + ERR=1 \ > + TARGET=${{ matrix.TARGET }} \ > + CC=${{ matrix.CC }} \ > + ${{ matrix.FLAGS }} \ > + ADDLIB="-Wl,-rpath,/usr/local/lib/ -Wl,-rpath,$HOME/opt/lib/" > + sudo make install > + - name: Show HAProxy version. > + id: show-version > + run: | > + echo "::group::Show dynamic libraries." > + if command -v ldd > /dev/null; then > + ldd $(which haproxy) > + else > + otool -L $(which haproxy) > + fi > + echo "::endgroup::" > + haproxy -vv > + echo "::set-output name=version::$(haproxy -v |awk 'NR==1{print > $3}')" > + - name: Run VTest for HAProxy ${{ steps.show-version.outputs.version > }} > + run: make reg-tests > -- > 2.28.0 > >