Hello Laszlo Gaal, Joe McDonnell,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20469
to look at the new patch set (#3).
Change subject: IMPALA-12436: Create multi-platform builder images
......................................................................
IMPALA-12436: Create multi-platform builder images
Updates the toolchain build image creation process to support publishing
multi-platform (aarch64 and amd64) images.
Uses docker buildx to produce multi-platform images. This requires QEMU
on the builder; these builds are quick, so the overhead of emulating arm
is minimal. Currently assumes images are built on amd64 as not all
builds specify `--platform`.
Moves the publishing step to happen within buildx, because buildx does
not support loading multi-platform images in the local Docker registry.
This does mean that a build failure could result in publishing only some
of the images. These are not updated frequently and are usually preceded
by a test run, so this risk seems minimal.
Python 2.7 ran into build issues installing awscli on ARM, so OS
targeting ARM were updated to ensure they include Python 3.8+ and
install a newer version of awscli in Python 3.
New options were added to buildall.py:
- registry: publish images to the specified registry
- multi: enables multi-platform builds on RedHat 8 and Ubuntu 20/22
Multi-platform builds require QEMU and a non-default buildx builder.
Jenkins job is updated to install QEMU, create a builder, and build
multi-platform images.
Multi-platform images can't be loaded into a local Docker context (where
they would be listed via docker images); we continue to load them there
by default with --load, but disable it when doing multi-platform builds.
The only way to get images out of the builder for multi-platform builds
is to specify a registry.
Change-Id: I058deb877f26b48a2d87a65710143bba9a42a5fb
---
M docker/all/postinstall.sh
M docker/buildall.py
M docker/redhat8.df
M docker/ubuntu2004.df
M docker/ubuntu2204.df
M jenkins/build.sh
6 files changed, 54 insertions(+), 21 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/native-toolchain
refs/changes/69/20469/3
--
To view, visit http://gerrit.cloudera.org:8080/20469
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I058deb877f26b48a2d87a65710143bba9a42a5fb
Gerrit-Change-Number: 20469
Gerrit-PatchSet: 3
Gerrit-Owner: Michael Smith <michael.sm...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <laszlo.g...@cloudera.com>
Gerrit-Reviewer: Michael Smith <michael.sm...@cloudera.com>
