On Fri, Apr 24, 2020 at 11:30 AM Daniel P. Berrangé <berra...@redhat.com> wrote: > > ----- Original Message ----- > > > From: "Daniel P. Berrangé" <berra...@redhat.com> > > > To: "Cleber Rosa" <cr...@redhat.com> [...] > > Hi Daniel, > > > > We're already using the shared x86 runners, but with a different goal. The > > goal of the "Gating CI" is indeed to expand on non-x86 environments. We're > > in a "chicken and egg" kind of situation, because we'd like to prove that > > GitLab CI will allow QEMU to expand to very different runners and jobs, > > while > > not really having all that hardware setup and publicly available at this > > time. > > > > My experiments were really around that point, I mean, confirming that we > > can grow > > the number of architectures/runners/jobs/configurations to provide a > > coverage > > equal or greater to what Peter already does. > > So IIUC, you're saying that for x86 gating, the intention is to use shared > runners in general. > > Your current work that you say is blocked on access to x86 hardware, is just > about demonstrating the concept of plugging in custom runners, while we wait > for access to non-x86 hardware ? > > > > With GitLab if someone forks the repo to their personal namespace, they > > > cannot use any custom runners setup by the origin project. So if we use > > > custom runners for x86, people forking won't be able to run the GitLab > > > CI jobs. > > > > > > > They will continue to be able use the jobs and runners already defined in > > the .gitlab-ci.yml file. This work will only affect people pushing to the/a > > "staging" branch. > > > > > As a sub-system maintainer I wouldn't like this, because I ideally want > > > to be able to run the same jobs on my staging tree, that Peter will run > > > at merge time for the PULL request I send. > > > > > > > If you're looking for symmetry between any PR and "merge time" jobs, the > > only solution is to allow any PR to access all the diverse set of non-shared > > machines we're hoping to have in the near future. This may be something > > we'll get to, but I doubt we can tackle it in the near future now. > > It occurred to me that we could do this if we grant selective access to > the Gitlab repos, to people who are official subsystem maintainers. > GitLab has a concept of "protected branches", so you can control who is > allowed to push changes on a per-branch granularity. > > So, for example, in the main qemu.git, we could create branches for each > subsystem tree eg > > staging-block > staging-qapi > staging-crypto > staging-migration > .... > > and for each of these branches, we can grant access to relevant subsystem > maintainer(s).
The MAINTAINERS file could help us with that, we already have scripts to parse its sections. Maintainers should keep it up-to-date, then the merge script would check, i.e.: <newline> // section separator --------------- // ignored Trivial patches // description ignored M: Michael Tokarev <m...@tls.msk.ru> M: Laurent Vivier <laur...@vivier.eu> // must match commit author T: git git://git.corpit.ru/qemu.git trivial-patches T: git https://github.com/vivier/qemu.git trivial-patches // must match MR source > > When they're ready to send a pull request to Peter, they can push their > tree to this branch. Since the branch is in the main gitlab.com/qemu/qemu > project namespace, this branch can run CI using the private QEMU runners. > The subsystem maintainer can thus see the full set of CI results across > all platforms required by Gating, before Peter even gets the pull request. > > So when Peter then looks at merging the pull request to master, the only > he's likely to see are the non-deterministic bugs, or issues caused by > semantic conflicts with other recently merged code. > > It would even be possible to do the final merge into master entirely from > GitLab, no need to go via email. When the source branch & target branch are > within the same git repo, GitLab has the ability to run CI jobs against the > resulting merge commit in a strict gating manner, before it hits master. > They call this "Merge trains" in their documentation. > > IOW, from Peter's POV, merging pull requests could be as simple as hitting > the merge button in GitLab merge request UI. Everything wrt CI would be > completely automated, and the subsystem maintainers would have the > responsibility to dealing with merge conflicts & CI failures, which is > more scalable for the person co-ordinating the merges into master. > > > Regards, > Daniel