El jue., 26 mar. 2020 a las 10:15, Martin Grigorov (< martin.grigo...@gmail.com>) escribió:
> Hello, > > Here is the PR: https://github.com/varnishcache/varnish-cache/pull/3263 > I will add some more documentation about the new setup. > Any feedback is welcome! > Nice work, Martin! Gracias! Emilio > > Regards, > Martin > > On Wed, Mar 25, 2020 at 9:55 PM Martin Grigorov <martin.grigo...@gmail.com> > wrote: > >> Hi, >> >> On Wed, Mar 25, 2020, 20:15 Guillaume Quintard < >> guilla...@varnish-software.com> wrote: >> >>> is that script running as root? >>> >> >> Yes. >> I also added 'USER root' to its Dockerfile and '-u 0' to 'docker run' >> arguments but it still doesn't work. >> The x86 build is OK. >> It must be something in the base docker image. >> I've disabled the Alpine aarch64 job for now. >> I'll send a PR tomorrow! >> >> Regards, >> Martin >> >> >>> -- >>> Guillaume Quintard >>> >>> >>> On Wed, Mar 25, 2020 at 2:30 AM Martin Grigorov < >>> martin.grigo...@gmail.com> wrote: >>> >>>> Hi, >>>> >>>> I've moved 'dist' job to be executed in parallel with 'tar_pkg_tools' >>>> and the results from both are shared in the workspace for the actual >>>> packing jobs. >>>> Now the new error for aarch64-apk job is: >>>> >>>> abuild: varnish >>> varnish: Updating the sha512sums in APKBUILD... >>>> ]0; DEBUG: 4 >>>> ]0;abuild: varnish >>> varnish: Building /varnish 6.4.0-r1 (using >>>> abuild 3.5.0-r0) started Wed, 25 Mar 2020 09:22:02 +0000 >>>> >>> varnish: Checking sanity of /package/APKBUILD... >>>> >>> WARNING: varnish: No maintainer >>>> >>> varnish: Analyzing dependencies... >>>> 0% % >>>> ############################################>>> varnish: Installing for >>>> build: build-base gcc libc-dev libgcc pcre-dev ncurses-dev libedit-dev >>>> py-docutils linux-headers libunwind-dev python py3-sphinx >>>> Waiting for repository lock >>>> ERROR: Unable to lock database: Bad file descriptor >>>> ERROR: Failed to open apk database: Bad file descriptor >>>> >>> ERROR: varnish: builddeps failed >>>> ]0; >>> varnish: Uninstalling dependencies... >>>> Waiting for repository lock >>>> ERROR: Unable to lock database: Bad file descriptor >>>> ERROR: Failed to open apk database: Bad file descriptor >>>> >>>> Google suggested to do this: >>>> rm -rf /var/cache/apk >>>> mkdir /var/cache/apk >>>> >>>> It fails at 'abuild -r' - >>>> https://github.com/martin-g/varnish-cache/blob/b62c357b389c0e1e31e9c001cbffb55090c2e49f/.circleci/make-apk-packages.sh#L61 >>>> >>>> Any hints ? >>>> >>>> Martin >>>> >>>> On Wed, Mar 25, 2020 at 2:39 AM Guillaume Quintard < >>>> guilla...@varnish-software.com> wrote: >>>> >>>>> Hi, >>>>> >>>>> So, you are pointing at the `dist` job, whose sole role is to provide >>>>> us with a dist tarball, so we don't need that command line to work for >>>>> everyone, just for that specific platform. >>>>> >>>>> On the other hand, >>>>> https://github.com/varnishcache/varnish-cache/blob/4f9d8bed6b24bf9ee900c754f37615fdba1c44db/.circleci/config.yml#L168 >>>>> is >>>>> closer to what you want, `distcheck` will be call on all platform, and you >>>>> can see that it has the `--with-unwind` argument. >>>>> -- >>>>> Guillaume Quintard >>>>> >>>>> >>>>> On Tue, Mar 24, 2020 at 3:05 PM Martin Grigorov < >>>>> martin.grigo...@gmail.com> wrote: >>>>> >>>>>> >>>>>> >>>>>> On Tue, Mar 24, 2020, 17:19 Guillaume Quintard < >>>>>> guilla...@varnish-software.com> wrote: >>>>>> >>>>>>> Compare your configure line with what's currently in use (or the >>>>>>> apkbuild file), there are a few options (with-unwind, without-jemalloc, >>>>>>> etc.) That need to be set >>>>>>> >>>>>> >>>>>> The configure line comes from "./autogen.des": >>>>>> https://github.com/varnishcache/varnish-cache/blob/4f9d8bed6b24bf9ee900c754f37615fdba1c44db/autogen.des#L35-L42 >>>>>> It is called at: >>>>>> >>>>>> https://github.com/varnishcache/varnish-cache/blob/4f9d8bed6b24bf9ee900c754f37615fdba1c44db/.circleci/config.yml#L40 >>>>>> In my branch at: >>>>>> >>>>>> https://github.com/martin-g/varnish-cache/blob/4b4626ee9cc366b032a45f27b54d77176125ef03/.circleci/make-apk-packages.sh#L26 >>>>>> >>>>>> It fails only on aarch64 for Alpine Linux. The x86_64 build for >>>>>> Alpine is fine. >>>>>> AARCH64 for CentOS 7 and Ubuntu 18.04 are also fine. >>>>>> >>>>>> Martin >>>>>> >>>>>> >>>>>>> On Tue, Mar 24, 2020, 08:05 Martin Grigorov < >>>>>>> martin.grigo...@gmail.com> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> On Tue, Mar 24, 2020 at 11:00 AM Martin Grigorov < >>>>>>>> martin.grigo...@gmail.com> wrote: >>>>>>>> >>>>>>>>> Hi Guillaume, >>>>>>>>> >>>>>>>>> On Mon, Mar 23, 2020 at 8:01 PM Guillaume Quintard < >>>>>>>>> guilla...@varnish-software.com> wrote: >>>>>>>>> >>>>>>>>>> Hi Martin, >>>>>>>>>> >>>>>>>>>> Thank you for that. >>>>>>>>>> A few remarks and questions: >>>>>>>>>> - how much time does the "docker build" step takes? We can >>>>>>>>>> possibly speed things up by push images to the dockerhub, as they >>>>>>>>>> don't >>>>>>>>>> need to change very often. >>>>>>>>>> >>>>>>>>> >>>>>>>>> Definitely such optimization would be a good thing to do! >>>>>>>>> At the moment, with 'machine' executor it fetches the base image >>>>>>>>> and then builds all the Docker layers again and again. >>>>>>>>> Here are the timings: >>>>>>>>> 1) Spinning up a VM - around 10secs >>>>>>>>> 2) prepare env variables - 0secs >>>>>>>>> 3) checkout code (varnish-cache) - 5secs >>>>>>>>> 4) activate QEMU - 2secs >>>>>>>>> 5) build packages >>>>>>>>> 5.1) x86 deb - 3m 30secs >>>>>>>>> 5.2) x86 rpm - 2m 50secs >>>>>>>>> 5.3) aarch64 rpm - 35mins >>>>>>>>> 5.4) aarch64 deb - 45mins >>>>>>>>> >>>>>>>>> >>>>>>>>>> - any reason why you clone pkg-varnish-cache in each job? The >>>>>>>>>> idea was to have it cloned once in tar-pkg-tools for consistency and >>>>>>>>>> reproducibility, which we lose here. >>>>>>>>>> >>>>>>>>> >>>>>>>>> I will extract the common steps once I see it working. This is my >>>>>>>>> first CircleCI project and I still find my ways in it! >>>>>>>>> >>>>>>>>> >>>>>>>>>> - do we want to change things for the amd64 platforms for the >>>>>>>>>> sake of consistency? >>>>>>>>>> >>>>>>>>> >>>>>>>>> So far there is nothing specific for amd4 or aarch64, except the >>>>>>>>> base Docker images. >>>>>>>>> For example make-deb-packages.sh is reused for both amd64 and >>>>>>>>> aarch64 builds. Same for -rpm- and now for -apk- (alpine). >>>>>>>>> >>>>>>>>> Once I feel the change is almost finished I will open a Pull >>>>>>>>> Request for more comments! >>>>>>>>> >>>>>>>>> Martin >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Guillaume Quintard >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Mon, Mar 23, 2020 at 6:25 AM Martin Grigorov < >>>>>>>>>> martin.grigo...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Wed, Mar 18, 2020 at 5:31 PM Martin Grigorov < >>>>>>>>>>> martin.grigo...@gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> On Thu, Mar 12, 2020 at 4:35 PM Martin Grigorov < >>>>>>>>>>>> martin.grigo...@gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Guillaume, >>>>>>>>>>>>> >>>>>>>>>>>>> On Thu, Mar 12, 2020 at 3:23 PM Guillaume Quintard < >>>>>>>>>>>>> guilla...@varnish-software.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Offering arm64 packages requires a few things: >>>>>>>>>>>>>> - arm64-compatible code (all good in >>>>>>>>>>>>>> https://github.com/varnishcache/varnish-cache) >>>>>>>>>>>>>> - arm64-compatible package framework (all good in >>>>>>>>>>>>>> https://github.com/varnishcache/pkg-varnish-cache) >>>>>>>>>>>>>> - infrastructure to build the packages (uhoh, see below) >>>>>>>>>>>>>> - infrastructure to store and deliver ( >>>>>>>>>>>>>> https://packagecloud.io/varnishcache) >>>>>>>>>>>>>> >>>>>>>>>>>>>> So, everything is in place, expect for the third point. At >>>>>>>>>>>>>> the moment, there are two concurrent CI implementations: >>>>>>>>>>>>>> - travis: >>>>>>>>>>>>>> https://github.com/varnishcache/varnish-cache/blob/master/.travis.yml >>>>>>>>>>>>>> It's >>>>>>>>>>>>>> the historical one, and currently only runs compilation+test for >>>>>>>>>>>>>> OSX >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Actually it tests Linux AMD64 and ARM64 too. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> - circleci: >>>>>>>>>>>>>> https://github.com/varnishcache/varnish-cache/blob/master/.circleci/config.yml >>>>>>>>>>>>>> the >>>>>>>>>>>>>> new kid on the block, that builds all the packages and >>>>>>>>>>>>>> distchecks for all >>>>>>>>>>>>>> the packaged platforms >>>>>>>>>>>>>> >>>>>>>>>>>>>> The issue is that cirecleci doesn't support arm64 containers >>>>>>>>>>>>>> (for now?), so we would need to re-implement the packaging logic >>>>>>>>>>>>>> in Travis. >>>>>>>>>>>>>> It's not a big problem, but it's currently not a priority on my >>>>>>>>>>>>>> side. >>>>>>>>>>>>>> >>>>>>>>>>>>>> However, I am totally ready to provide help if someone wants >>>>>>>>>>>>>> to take that up. The added benefit it that Travis would be able >>>>>>>>>>>>>> to handle >>>>>>>>>>>>>> everything and we can retire the circleci experiment >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I will take a look in the coming days and ask you if I need >>>>>>>>>>>>> help! >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I've took a look at the current setup and here is what I've >>>>>>>>>>>> found as problems and possible solutions: >>>>>>>>>>>> >>>>>>>>>>>> 1) Circle CI >>>>>>>>>>>> 1.1) problem - the 'machine' and 'Docker' executors run on >>>>>>>>>>>> x86_64, so there is no way to build the packages in a "native" >>>>>>>>>>>> environment >>>>>>>>>>>> 1.2) possible solutions >>>>>>>>>>>> 1.2.1) use multiarch cross build >>>>>>>>>>>> 1.2.2) use 'machine' executor that registers QEMU via >>>>>>>>>>>> https://hub.docker.com/r/multiarch/qemu-user-static/ and then >>>>>>>>>>>> builds and runs a custom Docker image that executes a shell script >>>>>>>>>>>> with the >>>>>>>>>>>> build steps >>>>>>>>>>>> It will look something like >>>>>>>>>>>> https://github.com/yukimochi-containers/alpine-vpnserver/blob/69bb0a612c9df3e4ba78064d114751b760f0df9d/.circleci/config.yml#L19-L38 >>>>>>>>>>>> but >>>>>>>>>>>> instead of uploading the Docker image as a last step it will run >>>>>>>>>>>> it. >>>>>>>>>>>> The RPM and DEB build related code from current config.yml will >>>>>>>>>>>> be extracted into shell scripts which will be copied in the custom >>>>>>>>>>>> Docker >>>>>>>>>>>> images >>>>>>>>>>>> >>>>>>>>>>>> From these two possible ways I have better picture in my head >>>>>>>>>>>> how to do 1.2.2, but I don't mind going deep in 1.2.1 if this is >>>>>>>>>>>> what you'd >>>>>>>>>>>> prefer. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I've decided to stay with Circle CI and use 'machine' executor >>>>>>>>>>> with QEMU. >>>>>>>>>>> >>>>>>>>>>> The changed config.yml could be seen at >>>>>>>>>>> https://github.com/martin-g/varnish-cache/tree/feature/aarch64-packages/.circleci >>>>>>>>>>> and >>>>>>>>>>> the build at >>>>>>>>>>> https://app.circleci.com/pipelines/github/martin-g/varnish-cache/71/workflows/3a275d79-62a9-48b4-9aef-1585de1c87c8 >>>>>>>>>>> The builds on x86 arch take 3-4 mins, but for aarch64 >>>>>>>>>>> (emulation!) ~40mins >>>>>>>>>>> For now the jobs just build the .deb & .rpm packages for CentOS >>>>>>>>>>> 7 and Ubuntu 18.04, both amd64 and aarch64. >>>>>>>>>>> TODOs: >>>>>>>>>>> - migrate Alpine >>>>>>>>>>> >>>>>>>>>> >>>>>>>> Build on Alpine aarch64 fails with: >>>>>>>> ... >>>>>>>> automake: this behaviour will change in future Automake versions: >>>>>>>> they will >>>>>>>> automake: unconditionally cause object files to be placed in the >>>>>>>> same subdirectory >>>>>>>> automake: of the corresponding sources. >>>>>>>> automake: project, to avoid future incompatibilities. >>>>>>>> parallel-tests: installing 'build-aux/test-driver' >>>>>>>> lib/libvmod_debug/Makefile.am:12: warning: libvmod_debug_la_LDFLAGS >>>>>>>> multiply defined in condition TRUE ... >>>>>>>> lib/libvmod_debug/automake_boilerplate.am:19: ... >>>>>>>> 'libvmod_debug_la_LDFLAGS' previously defined here >>>>>>>> lib/libvmod_debug/Makefile.am:9: 'lib/libvmod_debug/ >>>>>>>> automake_boilerplate.am' included from here >>>>>>>> + autoconf >>>>>>>> + CONFIG_SHELL=/bin/sh >>>>>>>> + export CONFIG_SHELL >>>>>>>> + ./configure '--prefix=/opt/varnish' '--mandir=/opt/varnish/man' >>>>>>>> --enable-maintainer-mode --enable-developer-warnings >>>>>>>> --enable-debugging-symbols --enable-dependency-tracking >>>>>>>> --with-persistent-storage --quiet >>>>>>>> configure: WARNING: dot not found - build will fail if svg files >>>>>>>> are out of date. >>>>>>>> configure: WARNING: No system jemalloc found, using system malloc >>>>>>>> configure: error: Could not find backtrace() support >>>>>>>> >>>>>>>> Does anyone know a workaround ? >>>>>>>> I use multiarch/alpine:aarch64-edge as a base Docker image >>>>>>>> >>>>>>>> Martin >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> - store the packages as CircleCI artifacts >>>>>>>>>>> - anything else that is still missing >>>>>>>>>>> >>>>>>>>>>> Adding more architectures would be as easy as adding a new >>>>>>>>>>> Dockerfile with a base image from the respective type. >>>>>>>>>>> >>>>>>>>>>> Martin >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> 2) Travis CI >>>>>>>>>>>> 2.1) problems >>>>>>>>>>>> 2.1.1) generally Travis is slower than Circle! >>>>>>>>>>>> Althought if we use CircleCI 'machine' executor it will be >>>>>>>>>>>> slower than the current 'Docker' executor! >>>>>>>>>>>> 2.1.2) Travis supports only Ubuntu >>>>>>>>>>>> Current setup at CircleCI uses CentOS 7. >>>>>>>>>>>> I guess the build steps won't have problems on Ubuntu. >>>>>>>>>>>> >>>>>>>>>>>> 3) GitHub Actions >>>>>>>>>>>> GH Actions does not support ARM64 but it supports self hosted >>>>>>>>>>>> ARM64 runners >>>>>>>>>>>> 3.1) The problem is that there is no way to make a self hosted >>>>>>>>>>>> runner really private. I.e. if someone forks Varnish Cache any >>>>>>>>>>>> commit in >>>>>>>>>>>> the fork will trigger builds on the arm64 node. There is no way to >>>>>>>>>>>> reserve >>>>>>>>>>>> the runner only for commits against >>>>>>>>>>>> https://github.com/varnishcache/varnish-cache >>>>>>>>>>>> >>>>>>>>>>>> Do you see other problems or maybe different ways ? >>>>>>>>>>>> Do you have preferences which way to go ? >>>>>>>>>>>> >>>>>>>>>>>> Regards, >>>>>>>>>>>> Martin >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Regards, >>>>>>>>>>>>> Martin >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> Guillaume Quintard >>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>> varnish-dev mailing list >>>>>>>>>>>>>> varnish-dev@varnish-cache.org >>>>>>>>>>>>>> >>>>>>>>>>>>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev >>>>>>>>>>>>>> >>>>>>>>>>>>>
_______________________________________________ varnish-dev mailing list varnish-dev@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev