On Fri, Sep 24, 2021 at 04:01:21PM +0200, Harald Seiler wrote: > Hi Simon, > > On Mon, 2021-09-20 at 08:06 -0600, Simon Glass wrote: > > Hi Harald, > > > > On Mon, 20 Sept 2021 at 02:12, Harald Seiler <h...@denx.de> wrote: > > > > > > Hi, > > > > > > On Sat, 2021-09-18 at 10:37 -0600, Simon Glass wrote: > > > > Hi, > > > > > > > > Is there something screwy with this? It seems that denx-vulcan does > > > > two builds at once? > > > > > > > > https://source.denx.de/u-boot/custodians/u-boot-dm/-/jobs/323540 > > > > > > Hm, I did some changes to the vulcan runner which might have caused > > > this... But still, even if it is running multiple jobs in parallel, they > > > should still be isolated, so how does this lead to a build failure? > > > > I'm not sure that it does, but I do see this at the above link: > > > > Error: Unable to create > > '/builds/u-boot/custodians/u-boot-dm/.git/logs/HEAD.lock': File > > exists. > > This is super strange... Each build should be running in its own > container so there should never be a way for such a race to occur. No > clue what is going on here...
I know this from having to track down a different oddball failure with konsulko-bootbake. It comes down to something along the lines of volumes being re-used. Good in that it means that every job every time isn't doing a whole clone of the u-boot tree. Bad in that just in case the job gets wedged/killed in a crazy spot you end up with problems like this. If you run a 'find' on vulcan you'll figure out which overlay has a problem. Or you can stop the runner for a moment and tell docker to purge unused volumes and it'll clear it up. > > Re doing multiple builds, have you set it up so it doesn't take on the > > very large builds? I would love to enable multiple builds for the qemu > > steps since they mostly use a single CPU, but am not sure how to do > > it. > > Actually, this was more a mistake than an intentional change. I updated > the runner on vulcan to also take jobs for some other repos and wanted > those jobs to run in parallel. It looks like I just forgot setting the > `limit = 1` option for the U-Boot runner. > > Now, I think doing what you suggest is possible. We need to tag build > and "test" jobs differently and then define multiple runners with > different limits. E.g. in `.gitlab-ci.yml`: > > build all 32bit ARM platforms: > stage: world build > tags: > - build > > cppcheck: > stage: testsuites > tags: > - test > > And then define two runners in `/etc/gitlab-runner/config.toml`: > > concurrent = 4 > > [[runners]] > name = "u-boot builder on vulcan" > limit = 1 > ... > > [[runners]] > name = "u-boot tester on vulcan" > limit = 4 > ... > > and during registration they get the `build` and `test` tags > respectively. This would allow running (in this example) up to 4 test > jobs concurrently, but only ever one large build job at once. Yes, but this would also make it harder for people to use the CI as-is with their own runners. For example, the only thing stopping people from using the free gitlab CI runners on their own is that squashfs test being broken. -- Tom
signature.asc
Description: PGP signature