Fabiano Rosas <faro...@suse.de> writes: > Alex Bennée <alex.ben...@linaro.org> writes: > >> Fabiano Rosas <faro...@suse.de> writes: >> >>> Our dockerfiles no longer reference layers from other qemu images so >>> we can now use 'docker build' on them. >>> >>> Also reinstate the caching that was disabled due to bad interactions >>> with certain runners. See commit 6ddc3dc7a8 ("tests/docker: don't use >>> BUILDKIT in GitLab either"). We now believe those issues to be fixed. >>> >>> The COMMON_TAG needed to be fixed for the caching to work. The >>> docker.py script was not using the variable, but constructing the >>> correct URL directly. >>> >>> Signed-off-by: Fabiano Rosas <faro...@suse.de> >>> --- >>> .gitlab-ci.d/container-template.yml | 9 ++++----- >>> 1 file changed, 4 insertions(+), 5 deletions(-) >>> >>> diff --git a/.gitlab-ci.d/container-template.yml >>> b/.gitlab-ci.d/container-template.yml >>> index c434b9c8f3..519b8a9482 100644 >>> --- a/.gitlab-ci.d/container-template.yml >>> +++ b/.gitlab-ci.d/container-template.yml >>> @@ -6,17 +6,16 @@ >>> - docker:dind >>> before_script: >>> - export TAG="$CI_REGISTRY_IMAGE/qemu/$NAME:latest" >>> - - export COMMON_TAG="$CI_REGISTRY/qemu-project/qemu/$NAME:latest" >>> + - export COMMON_TAG="$CI_REGISTRY/qemu-project/qemu/qemu/$NAME:latest" >>> - apk add python3 >>> - docker info >>> - docker login $CI_REGISTRY -u "$CI_REGISTRY_USER" -p >>> "$CI_REGISTRY_PASSWORD" >>> script: >>> - echo "TAG:$TAG" >>> - echo "COMMON_TAG:$COMMON_TAG" >>> - - ./tests/docker/docker.py --engine docker build >>> - -t "qemu/$NAME" -f "tests/docker/dockerfiles/$NAME.docker" >>> - -r $CI_REGISTRY/qemu-project/qemu >>> - - docker tag "qemu/$NAME" "$TAG" >>> + - docker build --tag "$TAG" --cache-from "$TAG" --cache-from >>> "$COMMON_TAG" >>> + --build-arg BUILDKIT_INLINE_CACHE=1 >>> + -f "tests/docker/dockerfiles/$NAME.docker" "." >> >> I wonder why this doesn't injest a bunch of context. If I run: >> >> docker build --cache-from >> registry.gitlab.com/stsquad/qemu/qemu/debian-alpha-cross --build-arg >> BUILDKIT_INLINE_CACHE=1 --build-arg USER=alex --build-arg UID=1000 -t qemu >> /debian-alpha-cross -f >> "/home/alex/lsrc/qemu.git/tests/docker/dockerfiles/debian-alpha-cross.docker" >> . >> >> it attempts to bring my entire build directory in as build context. This >> is why we use the - < docker form in the Makefile. >> > > I only see that without DOCKER_BUILDKIT=1. With the variable set it does > like in the CI jobs. I presume it is being set automatically by gitlab, > but we could add it to the script to be explicit. > >>> - docker push "$TAG" >>> after_script: >>> - docker logout >> >> So what I don't understand is if I do: >> >> docker pull registry.gitlab.com/stsquad/qemu/qemu/debian-alpha-cross >> docker build --cache-from >> registry.gitlab.com/stsquad/qemu/qemu/debian-alpha-cross --build-arg >> BUILDKIT_INLINE_CACHE=1 -t qemu/debian-alpha-cross - < >> >> /home/alex/lsrc/qemu.git/tests/docker/dockerfiles/debian-alpha-cross.docker >> >> I still see pretty much a full rebuild of the image. > > I don't use docker and podman does not support caching. I have > fresh-installed docker today and indeed it seems to not use the cache at > every build. We're missing something. > > Sometimes it works: >
Oops, sorry about the long lines, here it is again: === $ docker system prune -a -f Deleted build cache objects: xzbxmzaib3a8s0ufetop5ikhi 0ce9qln4ipd2vgf9xw9to0fdb se4hq3rce3lad20t9sqqnubob Total reclaimed space: 5.845kB $ docker images -a REPOSITORY TAG IMAGE ID CREATED SIZE $ DOCKER_BUILDKIT=1 docker build --tag registry.gitlab.com/farosas/qemu/qemu/debian-amd64:latest --cache-from registry.gitlab.com/farosas/qemu/qemu/debian-amd64:latest --build-arg BUILDKIT_INLINE_CACHE=1 -f debian-amd64.docker . [+] Building 57.8s (12/12) FINISHED => [internal] load build definition from debian-amd64.docker => => transferring dockerfile: 5.90kB => [internal] load .dockerignore => => transferring context: 2B => [internal] load metadata for docker.io/library/debian:11-slim => importing cache manifest from registry.gitlab.com/farosas/qemu/qemu/debian-amd64:latest => [1/6] FROM docker.io/library/debian:11-slim@sha256:8eaee63a5ea83744e62d5bf88e7d472d7f19b5feda3bfc6a2304cc074f269269 => CACHED [2/6] RUN export DEBIAN_FRONTEND=noninteractive && apt-get update && apt-get install -y eatmydata && eatmydata apt-get dist-upgrade -y && eatmydata apt-get i => CACHED [3/6] RUN DEBIAN_FRONTEND=noninteractive eatmydata apt install -y --no-install-recommends cscope global linux-headers-amd64 => CACHED [4/6] RUN git clone https://github.com/luigirizzo/netmap.git /usr/src/netmap => CACHED [5/6] RUN cd /usr/src/netmap && git checkout v11.3 => CACHED [6/6] RUN cd /usr/src/netmap/LINUX && ./configure --no-drivers --no-apps --kernel-dir=$(ls -d /usr/src/linux-headers-*-amd64) && make install => => pulling sha256:bb263680fed18eecdc67f885094df6f589bafc19004839d7fdf141df236a61aa => => pulling sha256:f63092d28e024c119d6a4e257d0c62bb2117be93605f1f2363b1d9cb8fc9dc6c => => pulling sha256:30fcedea9de9f22da43e20a79b40516c6e4a55e70eea401acacca18912e86769 => => pulling sha256:6f498daa6d20055a31e4681ec878e86129c3c1eaf5cf40f7ed87631accef1906 => => pulling sha256:79b1b0d7bce9a16b4a01e2055cd402d93e04efded50e2ffcf6be0d7c67e4895a => => pulling sha256:250b927c5ce2d6d3e9d24bf085f649ce951966c9ec9c9e00ab8292dfb02c8b17 => exporting to image => => exporting layers => => writing image sha256:3663223c0a31c2be678c2fe110ba284b80b8d04d2d3290c14be7a4d7320f8a67 => => naming to registry.gitlab.com/farosas/qemu/qemu/debian-amd64:latest => exporting cache => => preparing build cache for export === > > There are issues pointing to some general unreliability of the cache: > > https://github.com/moby/buildkit/issues/2274 > https://github.com/moby/buildkit/issues/1981 > > What is your docker version? Mine is 20.10.17-ce and Gitlab's is > 23.0.1. The issues above are on 20.10.11.