On 2/23/21 6:24 PM, Philippe Mathieu-Daudé wrote: > On 2/23/21 5:47 PM, Cleber Rosa wrote: >> On Tue, Feb 23, 2021 at 05:37:04PM +0100, Philippe Mathieu-Daudé wrote: >>> On 2/23/21 12:25 PM, Thomas Huth wrote: >>>> On 19/02/2021 22.58, Cleber Rosa wrote: >>>>> As described in the included documentation, the "custom runner" jobs >>>>> extend the GitLab CI jobs already in place. One of their primary >>>>> goals of catching and preventing regressions on a wider number of host >>>>> systems than the ones provided by GitLab's shared runners. >>>>> >>>>> This sets the stage in which other community members can add their own >>>>> machine configuration documentation/scripts, and accompanying job >>>>> definitions. As a general rule, those newly added contributed jobs >>>>> should run as "non-gating", until their reliability is verified (AKA >>>>> "allow_failure: true"). >>>>> >>>>> Signed-off-by: Cleber Rosa <cr...@redhat.com> >>>>> --- >>>>> .gitlab-ci.d/custom-runners.yml | 14 ++++++++++++++ >>>>> .gitlab-ci.yml | 1 + >>>>> docs/devel/ci.rst | 28 ++++++++++++++++++++++++++++ >>>>> docs/devel/index.rst | 1 + >>>>> 4 files changed, 44 insertions(+) >>>>> create mode 100644 .gitlab-ci.d/custom-runners.yml >>>>> create mode 100644 docs/devel/ci.rst >>>>> >>>>> diff --git a/.gitlab-ci.d/custom-runners.yml >>>>> b/.gitlab-ci.d/custom-runners.yml >>>>> new file mode 100644 >>>>> index 0000000000..3004da2bda >>>>> --- /dev/null >>>>> +++ b/.gitlab-ci.d/custom-runners.yml >>>>> @@ -0,0 +1,14 @@ >>>>> +# The CI jobs defined here require GitLab runners installed and >>>>> +# registered on machines that match their operating system names, >>>>> +# versions and architectures. This is in contrast to the other CI >>>>> +# jobs that are intended to run on GitLab's "shared" runners. >>>>> + >>>>> +# Different than the default approach on "shared" runners, based on >>>>> +# containers, the custom runners have no such *requirement*, as those >>>>> +# jobs should be capable of running on operating systems with no >>>>> +# compatible container implementation, or no support from >>>>> +# gitlab-runner. To avoid problems that gitlab-runner can cause while >>>>> +# reusing the GIT repository, let's enable the recursive submodule >>>>> +# strategy. >>>>> +variables: >>>>> + GIT_SUBMODULE_STRATEGY: recursive >>>> >>>> Is it really necessary? I thought our configure script would take care >>>> of the submodules? >>> >> >> I've done a lot of testing on bare metal systems, and the problems >> that come from reusing the same system and failed cleanups can be very >> frustrating. It's unfortunate that we need this, but it was the >> simplest and most reliable solution I found. :/ >> >> Having said that, I noticed after I posted this series that this is >> affecting all other jobs. We don't need it that in the jobs based >> on containers (for obvious reasons), so I see two options: >> >> 1) have it enabled on all jobs for consistency >> >> 2) have it enabled only on jobs that will reuse the repo >> >>> Well, if there is a failure during the first clone (I got one network >>> timeout in the middle) > > [This network failure is pasted at the end] > >>> then next time it doesn't work: >>> >>> Updating/initializing submodules recursively... >>> Synchronizing submodule url for 'capstone' >>> Synchronizing submodule url for 'dtc' >>> Synchronizing submodule url for 'meson' >>> Synchronizing submodule url for 'roms/QemuMacDrivers' >>> Synchronizing submodule url for 'roms/SLOF' >>> Synchronizing submodule url for 'roms/edk2' >>> Synchronizing submodule url for >>> 'roms/edk2/ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3' >>> Synchronizing submodule url for >>> 'roms/edk2/BaseTools/Source/C/BrotliCompress/brotli' >>> Synchronizing submodule url for >>> 'roms/edk2/BaseTools/Source/C/BrotliCompress/brotli/research/esaxx' >>> Synchronizing submodule url for >>> 'roms/edk2/BaseTools/Source/C/BrotliCompress/brotli/research/libdivsufsort' >>> Synchronizing submodule url for >>> 'roms/edk2/CryptoPkg/Library/OpensslLib/openssl' >>> Synchronizing submodule url for >>> 'roms/edk2/MdeModulePkg/Library/BrotliCustomDecompressLib/brotli' >>> Synchronizing submodule url for >>> 'roms/edk2/MdeModulePkg/Universal/RegularExpressionDxe/oniguruma' >>> Synchronizing submodule url for >>> 'roms/edk2/UnitTestFrameworkPkg/Library/CmockaLib/cmocka'
So far, beside the repository useful for QEMU, I cloned: - boringssl - krb5 - pyca-cryptography - esaxx - libdivsufsort - oniguruma - openssl - brotli - cmocka But reach the runner time limit of 2h. The directory reports 3GB of source code. I don't think the series has been tested enough before posting, I'm stopping here my experiments. Regards, Phil.