This all ties into the versioning of artifacts since you can’t have concurrent builds of the same repo (for example from 2 distinct gerrit triggers on different patchsets) without proper tagging of the artifacts (you can’t just tag them with “latest”)
I think it would make sense to allow only 1 build at a time per repo but to support multiple concurrent container builds from different repos. Timing out and the part of the script that checks in a loop if another build is ongoing should be removed IMHO since the max concurrency is better checked at the Jenkins level. Given hardware nowadays have a lot of cores, it would be good to allow multiple queues (or slaves) per hardware (up to 1 per core). Repos that build more than 1 container should have a way to provide the build order if they require one. Alec From: <opnfv-tech-discuss-boun...@lists.opnfv.org> on behalf of "Beierl, Mark" <mark.bei...@dell.com> Date: Monday, September 25, 2017 at 11:47 AM To: "opnfv-tech-discuss@lists.opnfv.org" <opnfv-tech-discuss@lists.opnfv.org> Subject: [opnfv-tech-discuss] Many failed docker builds Hello, Not sure who can help with this. Right now there are 4 or more executor slots than can execute any given opnfv-docker.sh script. The problem is with so many docker jobs being introduced in Euphrates, we are getting a lot of failures due to timeouts. For example, on arm-build4, 4 docker builds can start in parallel, but only one will pass the build in progress check. The other 3 jobs will wait for the first build to complete, and then the next will start. If the total wait time for any of the builds exceeds 30 minutes, we get a failure. This is happening more frequently. There are a couple options that I can see: * Reduce the number of executors to 1. This might have unintended side-effects on build times for other jobs. * Reduce the number of executors to 1 and add more slaves for the given hardware (both ARM and x86) * Fix the opnfv-docker.sh script so that it can allow more than one docker build to execute simultaneously. I don't know what this would encompass. I would really love to hear other's opinions on what can be done. Regards, Mark Mark Beierl SW System Sr Principal Engineer Dell EMC | Office of the CTO mobile +1 613 314 8106<tel:1-613-314-8106> mark.bei...@dell.com<mailto:mark.bei...@dell.com>
_______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
