On Fri, Jan 03, 2020 at 02:29:07PM +0000, Wei Liu wrote: > On Thu, Dec 19, 2019 at 02:42:17PM +0000, Anthony PERARD wrote: > > GitLab have a caching capability, see [1]. Let's use it to avoid using > > Internet too often. > > > > The cache is setup so that when xen.git/Config.mk is changed, the > > cache will need to be recreated. This has been chosen because that is > > where the information about how to clone sub-project trees is encoded > > (revisions). That may not work for qemu-xen tree which usually is > > `master', but that should be fine for now. > > > > The cache is populated of "git bundle" which will contain a mirror of > > the original repo, and can be cloned from. If the bundle exist, the > > script have the Xen makefiles clone from it, otherwise it will clone > > from the original URL and the bundles will be created just after. > > > > We have more than one runner in GitLab, and no shared cache between > > them, so every build jobs will be responsible to create the cache. > > > > [1] https://docs.gitlab.com/ee/ci/yaml/README.html#cache > > > > Signed-off-by: Anthony PERARD <anthony.per...@citrix.com> > > This is a good improvement. > > Have you run this in Gitlab CI? Can you point me to a run?
I have use the CI to develop the patch, so yes I have a run of it. But it is a run made with my wip branch, still it should be the same result if it was done with the final patch: https://gitlab.com/xen-project/people/anthonyper/xen/pipelines/104343621 > > diff --git a/automation/scripts/prepare-cache.sh > > b/automation/scripts/prepare-cache.sh > > new file mode 100755 > > index 000000000000..017f1b8f0672 > > --- /dev/null > > +++ b/automation/scripts/prepare-cache.sh > > @@ -0,0 +1,52 @@ > > +#!/bin/bash > > + > > +set -ex > > + > > +cachedir="${CI_PROJECT_DIR:=`pwd`}/ci_cache" > > +mkdir -p "$cachedir" > > + > > +declare -A r > > +r[extras/mini-os]=MINIOS_UPSTREAM_URL > > +r[tools/qemu-xen-dir]=QEMU_UPSTREAM_URL > > +r[tools/qemu-xen-traditional-dir]=QEMU_TRADITIONAL_URL > > +r[tools/firmware/ovmf-dir]=OVMF_UPSTREAM_URL > > +r[tools/firmware/seabios-dir]=SEABIOS_UPSTREAM_URL > > Does this mean if in the future we add or remove trees we will need to > modify this part in the same commit? We would need to modify the script when trees are removed, because I haven't thought of that. But when trees are added, the script can be changed in a follow-up. Ideally, we would use the Makefiles to discovers the git clones that can be cached, but that's not possible just yet. In the mean time, I think I should make the script more robust against removal of trees, so it doesn't have to be modified in the same commit. Thanks, -- Anthony PERARD _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
