On Mon, 16 Jan 2023 at 18:41, Oliver Steffen <ostef...@redhat.com> wrote: > > Update CI, run all Linux (aka Ubuntu-GCC5) based jobs in custom > containers. This decouples the CI environment from the virtual machine > images that Azure DevOps provides. The currently used ubuntu-18.04 image > has been deprecated for a while now and will be removed soon. > > The container image provides the required compiler toolchains and Qemu > for the supported architectures. These are then no longer downloaded at > runtime, avoiding CI failures due to download errors. This approach also > makes it easier to switch to other or newer compilers. It makes the CI > setup independent from the default images that Azure DevOps provides. > It can also help debugging CI problems, because the CI environment > can be reproduced on a local machine. > > The container images are hosted on ghcr.io and are automatically > generated using GitHub Actions. The Dockerfiles are maintained in the > Tianocore "containers" repository: > https://github.com/tianocore/containers. > > The current image is based on Fedora 35, with gcc 11. Fedora was chosen > because of its fast release cycle which makes it easy to keep the > toolchains up-to-date. > > Some further possible changes not included in this series: > - The Tianocore/containers repository provides stack of layered images. > One image for general purpose (build+test) and build-only jobs. > The build+test image is based on the build-only one and adds Qemu, > for the testing job that involve Qemu. The work in the image side > is done, we just need to change the CI setup accordingly. > This patch set uses the build+test images for all jobs. > - Further reduce the number of external dependencies that need to be > downloaded at runtime. Candidates are iasl and nasm, which are already > included in the image but not used yet. > > PR: https://github.com/tianocore/edk2/pull/3904 > > v11: > - Rebase to current master branch > - Use lasted image, includes Qemu 7.2 > - Set container image URL in a central place (the defaults.yml template) > > v10: > - Split commits by package > - Use ubuntu-22.04 as vm_image (instead of the floating ubuntu-latest) > - Some fixups around the Python version template argument and defaults.yml > - Fix some commit messages > - Add license statement to defaults.yml > > v9: > - Drop the "Don't install cspell" patch > - Use explicitly use ubuntu-22.04 as vm_image > > v8: > - Use updated container image that contains gcc for LoongArch64. > - Remove ext_dep files for the LoongArch64 gcc. > - Don't change the scopes in .pytool/CISettings.py if running Linux CI > it. > - Split commits that touch multiple packages. > - Use the smaller "build" image for jobs that allow it. > - Add a CI template file as a central place to define the default Python > version > and use it where needed. > > v7: > - Rebase to latest master branch. > - Use latest Fedora 35 CI image. > - Stop using the ubuntu-18.04 vm_image since this will no longer be available > after Dec 1st. Use ubuntu-latest instead. > > v6: > - Include suggestions by Chris Fernald. > - Added a parameter for the container image to the job template, makes usage > of containers optional. > - Added a parameter to configure the Python version to download. Allows > using Python from the VM/container image also. > - Restructure the commits (no further functional changes). > > v5: > - Update image > > v4: > - Use the latest image from the tianocode/containers repository which > - does not include acpica-tools > - includes Pyhton 3.10 > > v3: > - Use the latest image from the tianocode/containers repository which > pins down version numbers of gcc, iasl, and nasm in the Dockerfile. > > v2: > - Images are now hosted under the Tianocore Organization > https://github.com/tianocore/containers > > v1: > - Thread: https://edk2.groups.io/g/devel/message/89058 > - Images were hosted at https://github.com/osteffenrh/edk2-build-images > > Signed-off-by: Oliver Steffen <ostef...@redhat.com> > Acked-by: Ard Biesheuvel <a...@kernel.org> > Acked-by: Gerd Hoffmann <kra...@redhat.com> > Reviewed-by: Chris Fernald <chfer...@microsoft.com> > > Oliver Steffen (17): > CI: make Python version configurable > ArmVirtPkg: CI: use Python version from defaults template > EmulatorPkg: CI: use Python version from defaults template > OvmfPkg: CI: use Python version from defaults template > CI: add ~/.local/bin to PATH (Linux only) > CI: Allow running in a container. > CI: Use Fedora 35 container (Linux only) > ArmVirtPkg: CI: Use Fedora 35 container (Linux only) > EmulatorPkg: CI: Use Fedora 35 container (Linux only) > OvmfPkg: CI: Use Fedora 35 container (Linux only) > .pytool: CISettings.py: don't add scopes for GCC > BaseTools: remove ext_dep files for gcc > ArmVirtPkg: CI: use ubuntu-22.04 vm_image (Linux only) > EmulatorPgk: CI: use ubuntu-22.04 vm_image (Linux only) > OvmfPkg: CI: use ubuntu-22.04 vm_image (Linux only) > CI: use ubuntu-22.04 image (Linux only)
Thanks I have created a PR here https://github.com/tianocore/edk2/pull/3913 with all the acks added. > OfmvPkg: CI: Qemu tests: set CPU to Broadwell > ... and I omitted this one (as discussed on the thread) > .azurepipelines/Ubuntu-GCC5.yml | 12 +++++----- > .azurepipelines/Windows-VS2019.yml | 4 ++++ > .../templates/basetools-build-steps.yml | 9 -------- > .azurepipelines/templates/defaults.yml | 12 ++++++++++ > .../templates/platform-build-run-steps.yml | 12 +++++++++- > .../templates/pr-gate-build-job.yml | 6 +++++ > .azurepipelines/templates/pr-gate-steps.yml | 12 ++++++++-- > .pytool/CISettings.py | 9 -------- > .../.azurepipelines/Ubuntu-GCC5.yml | 12 +++++----- > BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml | 21 ------------------ > BaseTools/Bin/gcc_arm_linux_ext_dep.yaml | 21 ------------------ > ...gcc_loongarch64_unknown_linux_ext_dep.yaml | 22 ------------------- > .../Bin/gcc_riscv64_unknown_ext_dep.yaml | 22 ------------------- > .../.azurepipelines/Ubuntu-GCC5.yml | 9 +++++++- > .../.azurepipelines/Windows-VS2019.yml | 4 ++++ > .../.azurepipelines/Ubuntu-GCC5.yml | 12 +++++----- > .../.azurepipelines/Windows-VS2019.yml | 5 +++++ > OvmfPkg/PlatformCI/PlatformBuildLib.py | 1 + > 18 files changed, 82 insertions(+), 123 deletions(-) > create mode 100644 .azurepipelines/templates/defaults.yml > delete mode 100644 BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml > delete mode 100644 BaseTools/Bin/gcc_arm_linux_ext_dep.yaml > delete mode 100644 BaseTools/Bin/gcc_loongarch64_unknown_linux_ext_dep.yaml > delete mode 100644 BaseTools/Bin/gcc_riscv64_unknown_ext_dep.yaml > > -- > 2.39.0 > -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#98727): https://edk2.groups.io/g/devel/message/98727 Mute This Topic: https://groups.io/mt/96311815/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-