On Fri, Mar 19, 2021 at 12:27:10PM -0300, Wainer dos Santos Moschetta wrote: > Hi, > > On 3/19/21 8:34 AM, Philippe Mathieu-Daudé wrote: > > On 3/19/21 11:59 AM, Paolo Bonzini wrote: > > > On 19/03/21 11:18, Andrew Jones wrote: > > > > > Yikes, that is 41 hours per CI run. I wonder if GitLab's CI minutes > > > > > are > > > > > on slow machines or if we'll hit the same issue with dedicated > > > > > runners. > > > > > It seems like CI optimization will be necessary... > > > > > > > > > We need to reduce the amount of CI we do, not only because we can't > > > > afford > > > > it, but because it's wasteful. I hate to think of all the kWhs spent > > > > testing the exact same code in the exact same way, since everyone runs > > > > everything with a simple 'git push'. > > > Yes, I thought the same. > > > > > > > IMHO, 'git push' shouldn't trigger > > > > anything. Starting CI should be an explicit step. > > * tests/acceptance: Only run tests tagged 'gating-ci' on GitLab CI > > https://www.mail-archive.com/qemu-devel@nongnu.org/msg756464.html > > > > * gitlab-ci: Allow forks to select & restrict build jobs > > https://www.mail-archive.com/qemu-devel@nongnu.org/msg758331.html > > In my opinion that series is the first step towards a smart CI. It got some > reviews of Thomas and myself already but it didn't move ahead. If Philippe > for some reason cannot continue that work, I'm volunteering to take it over.
Hi, I wanted to follow up with a summary of the CI jobs: 1. Containers & Containers Layer2 - ~3 minutes/job x 39 jobs 2. Builds - ~50 minutes/job x 61 jobs 3. Tests - ~12 minutes/job x 20 jobs 4. Deploy - 52 minutes x 1 job The Builds phase consumes the most CI minutes. If we can optimize this phase then we'll achieve the biggest impact. In the short term builds could be disabled. However, in the long term I think full build coverage is desirable to prevent merging code that breaks certain host OSes/architectures (e.g. stable Linux distros, macOS, etc). Traditionally ccache (https://ccache.dev/) was used to detect recompilation of the same compiler input files. This is trickier to do in GitLab CI since it would be necessary to share and update a cache, potentially between untrusted users. Unfortunately this shifts the bottleneck from CPU to network in a CI-as-a-Service environment since the cached build output needs to be accessed by the linker on the CI runner but is stored remotely. A complementary approach is avoiding compilation altogether when code changes do not affect a build target. For example, a change to qemu-storage-daemon.c does not require rebuilding the system emulator targets. Either the compiler or the build system could produce a manifest of source files that went into a build target, and that information is what's needed to avoid compiling unchanged targets. Ideally the CI would look at the code changes and only launch jobs that were affected. Those jobs would use a C compiler cache to avoid rebuilding compiler input that has not changed. Basically, we need incremental builds. This is as far as I've gotten with thinking about CI efficiency. Do you think these optimizations are worth investigating or should we keep it simple and just disable many builds by default? Stefan
signature.asc
Description: PGP signature
