On Wed, Oct 18, 2017 at 09:35:49AM +0100, Daniel P. Berrange wrote: > On Wed, Oct 18, 2017 at 04:28:36PM +0800, Fam Zheng wrote: > > On Wed, 10/18 01:08, no-re...@patchew.org wrote: > > > Your branch is up-to-date with 'origin/test'. > > > Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path > > > 'dtc' > > > Cloning into > > > '/var/tmp/patchew-tester-tmp-jhpnlaz0/src/docker-src.2017-10-18-04.03.32.32287/qemu.tar.vroot/dtc'... > > > fatal: unable to connect to git.qemu-project.org: > > > git.qemu-project.org[0: 172.99.69.163]: errno=Connection timed out > > > > > > fatal: clone of 'git://git.qemu-project.org/dtc.git' into submodule path > > > '/var/tmp/patchew-tester-tmp-jhpnlaz0/src/docker-src.2017-10-18-04.03.32.32287/qemu.tar.vroot/dtc' > > > failed > > > Failed to clone 'dtc'. Retry scheduled > > > Cloning into > > > '/var/tmp/patchew-tester-tmp-jhpnlaz0/src/docker-src.2017-10-18-04.03.32.32287/qemu.tar.vroot/dtc'... > > > fatal: unable to connect to git.qemu-project.org: > > > git.qemu-project.org[0: 172.99.69.163]: errno=Connection timed out > > > > > > fatal: clone of 'git://git.qemu-project.org/dtc.git' into submodule path > > > '/var/tmp/patchew-tester-tmp-jhpnlaz0/src/docker-src.2017-10-18-04.03.32.32287/qemu.tar.vroot/dtc' > > > failed > > > Failed to clone 'dtc' a second time, aborting > > > failed to init submodule dtc > > > > So now patchew is less robust on network hiccups: a number of the same "git > > submodule update --init" calls have succeeded above, so this is incidental. > > Any > > idea how to improve that? Maybe if there is a checkout from $QEMU_SRC, > > clone/cp > > from there instead of fetch from submodule remote? > > If patchew has unreliable connectivity to git.qemu.org then the best thing > is probably have a background job that mirrors the git.qemu.org repos on > the patchew server. Then have the test job modify .gitmodules to point to > your local mirror. That way, network access is used during the background > job only, and actual builds just get a local clone from the cache.
I thought of a simpler way to address this. Currently patchew runs 4 docker jobs, and each one triggers creation of the $(DOCKER_SRC_COPY) rule. This runs archive-source.sh 4 times, because $(DOCKER_SRC_COPY) contains the current timestamp and so is different each time we run it. If you override DOCKER_SRC_COPY with the current git change hash, then you would end up using the same archive for all 4 jobs, and so only do the checkouts once make docker-foo DOCKER_SRC_COPY=docker-src.`git rev-parse HEAD` Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
