On Thu, Mar 02, 2023 at 06:48:35PM -0800, Stefano Stabellini wrote: > On Thu, 2 Mar 2023, Anthony PERARD wrote: > > Patch series available in this git branch: > > https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git > > br.gitlab-containers-auto-rebuild-v1 > > > > Hi, > > > > I have done some research to be able to build containers in the CI. This > > works > > only for x86 containers as I've setup only a single x86 gitlab-runner to be > > able to run docker-in-docker. > > > > The runner is setup to only accept jobs from a branch that is "protected" in > > gitlab. Also, one need credential to push to the container register, those > > are > > called "Group deploy tokens", and I've set the variables CI_DEPLOY_USER and > > CI_DEPLOY_PASSWORD in the project "xen-project/xen" (variables only visible > > on > > a pipeline running on a protected branch). > > > > These patch introduce quite a lot of redundancies in jobs, 2 new jobs per > > containers which build/push containers, and duplicate most of build.yaml. > > This mean that if we go with this, we have to duplicate and keep in sync > > many > > jobs. > > > > To avoid having to do the duplicated jobs by hand, I could look at > > creating a script that use "build.yaml" as input and produce the 3 > > stages needed to update a container, but that script would need to be > > run by hand, as gitlab can't really use it, unless .. > > > > I've look at generated pipeline, and they look like this in gitlab: > > > > https://gitlab.com/xen-project/people/anthonyper/xen/-/pipelines/777665738 > > But the result of the generated/child pipeline doesn't seems to be taken > > into > > account in the original pipeline, which make me think that we can't use > > them to > > generate "build.yaml". But maybe the could be use for generating the > > pipeline > > that will update a container. > > Doc: > > > > https://docs.gitlab.com/ee/ci/pipelines/downstream_pipelines.html#dynamic-child-pipelines > > > > So, with all of this, is it reasonable to test containers before > > pushing them to production? Or is it to much work? We could simply have jobs > > tacking care of rebuilding a container and pushing them to production > > without > > testing. > > I don't think it is a good idea to duplicate build.yaml, also because > some of the containers are used in the testing stage too, so an updated > container could be OK during the build phase and break the testing > phase. We would need to duplicate both build.yaml and test.yaml, which > is not feasible. > > In practice today people either: > 1) re-build a container locally & test it locally before pushing > 2) re-build a container locally, docker push it, then run a private > gitlab pipeline, if it passes send out a patch to xen-devel > > 1) is not affected by this series > 2) is also not affected because by the time the pipeline is created, the > container is already updated > > However, there are cases where it would definitely be nice to have a > "button" to press to update a container. For instance, when a pipeline > failis due to a Debian unstable apt-get failure, which can be easily fixed > by updating the Debian unstable container. > > So I think it would be nice to have jobs that can automatically update > the build containers but I would set them to manually trigger instead of > automatically (when: manual).
What I was looking at with this work was to be able to have container been rebuild automatically on a schedule. We have/had containers that were 3yr old, and when it's a container that supposed to test the lasted version of a distro, or a rolling release distro, they are kind of useless if they aren't rebuild regularly. So I was looking to take the human out of the loop and have computers the tedious work of rebuilding a container every month or two. Containers that needs to be rebuilt regularly to stay relevant are archlinux, debian/unstable, fedora/latest, propably opensuse/leap and opensuse/tumbleweed. I don't know if they are others. > Alternatively, we could move the "containers.yaml" stage to be the first > step, rebuild the containers and push them to a "staging" area > (registry.gitlab.com/xen-project/xen/staging), run the build and test > steps fetching from the staging area instead of the regular, if all > tests pass, then push the containers to > registry.gitlab.com/xen-project/xen as last step. Sounds good, I can look into how easily it would be to use a different registry to run a pipeline. Cheers, -- Anthony PERARD
