TL;DR: Patch 13 is an experiment to dynamically generate pipelines. Hi,
This series is an intent to allow git forks to use different set of jobs than the ones used by the mainstream repository. Currently, a fork gets the mainstream CI pipeline YAML configuration and runs its set of jobs by default. Mainstream tests many cases which might not be always useful to forks. Beside, one mainstream pipeline might be a waste of ressources when forks provide their own runners to their gitlab namespace, or use private instances. This is not a new / unique problem to our community, and Gitlab already figured it out. They describe this case as Upstream (our project mainline) / Downstream (a fork). The pipeline is divided in 2 steps. First the "upstream" part, which does some evaluation, and depending on the result, triggers the "downstream" part. The "upstream" is committed into mainline, and dynamically generate the "downstream" pipeline. This allows forks to easily rebase with mainstream, avoiding merge conflicts. This might not be adapted to the QEMU use case, but I wanted to give it a try. Gitlab recommends to use job templates. To be able to re-use / include templates, they can not use YAML anchors. All current jobs has first to be converted to use the 'extends' keyworkd. This is done by patches 1-5 (already reviewed). Patches 6-12 extract various generic jobs (which can be reused by forks) as template. Patches 13-14 follow Gitlab guideline to generate dynamic pipelines Patch 15 is an example of how a fork could use mainstream templates to maintain his own set of jobs. The PRO is forks can contribute to templates. Regards, Phil. Philippe Mathieu-Daudé (15): gitlab-ci: Replace YAML anchors by extends (container_job) gitlab-ci: Replace YAML anchors by extends (native_build_job) gitlab-ci: Replace YAML anchors by extends (native_test_job) gitlab-ci: Replace YAML anchors by extends (acceptance_test_job) gitlab-ci: Rename acceptance_test_job -> integration_test_job gitlab-ci: Extract container job template to container-template.yml gitlab-ci: Extract crossbuild job templates to crossbuild-template.yml gitlab-ci: Extract DCO/style check jobs to checks.yml gitlab-ci: Extract build stages to stages.yml gitlab-ci: Extract all default build/test jobs to buildtest.yml gitlab-ci: Extract core container jobs to container-core.yml gitlab-ci: Move current job set to qemu-project.yml gitlab-ci: Switch to dynamically generated pipelines gitlab-ci: Allow forks to use different set of jobs gitlab-ci: Use my own set of jobs for CI pipeline .gitlab-ci.d/buildtest-template.yml | 80 +++ .gitlab-ci.d/buildtest.yml | 744 +++++++++++++++++++++++ .gitlab-ci.d/checks.yml | 24 + .gitlab-ci.d/container-core.yml | 17 + .gitlab-ci.d/container-template.yml | 22 + .gitlab-ci.d/containers.yml | 108 ++-- .gitlab-ci.d/crossbuild-template.yml | 41 ++ .gitlab-ci.d/crossbuilds.yml | 43 +- .gitlab-ci.d/philmd.yml | 33 ++ .gitlab-ci.d/qemu-project.yml | 8 + .gitlab-ci.d/stages.yml | 8 + .gitlab-ci.yml | 845 +-------------------------- 12 files changed, 1033 insertions(+), 940 deletions(-) create mode 100644 .gitlab-ci.d/buildtest-template.yml create mode 100644 .gitlab-ci.d/buildtest.yml create mode 100644 .gitlab-ci.d/checks.yml create mode 100644 .gitlab-ci.d/container-core.yml create mode 100644 .gitlab-ci.d/container-template.yml create mode 100644 .gitlab-ci.d/crossbuild-template.yml create mode 100644 .gitlab-ci.d/philmd.yml create mode 100644 .gitlab-ci.d/qemu-project.yml create mode 100644 .gitlab-ci.d/stages.yml -- 2.26.3
