I have 3 build slaves and ~100 jobs:. * about ~80 jobs (it varies day-to-day), that require exclusive access to the file system, and hence cannot be run at the same time, on the same node. * about 20 jobs, that do not use the filesystem, and can be run in parallel
Since most jobs require exclusive access to the filesystem, the way I've handled the exclusive access is to only allow each Jenkins slave to run one job at a time. This works, but is inefficient, since the slaves could happily run the parallelisable jobs. I'd like to be able to run: * at most one of the 80 jobs per node at the same time * at most 9 of the 20 jobs per node at the same time I could do this using the Throttle Concurrent Jobs Plugin, by putting the 80 jobs in a "needs-exclusive-access" category, and setting the max number of concurrent jobs for that category per node to be 1, and increasing the number of slave executions to, say, 9. The problem with this solution is that when a new job is created, which will likely require exclusive access to the filesystem, we will inevitably sometimes forget to add the job to the "needs-exclusive-access", and then it'll start executing when other jobs are executing. Can anyone think of a solution that that allows jobs to by default be part of a "needs-exclusive-access" category, but be able to opt into not needing exclusive access? I realise the root problem is that jobs require exclusive access. I plan to fix this, but it will take time and training the rest of the team. I'm looking for a solution that will work in the interim. Regards, Tom PS. I tried the Heavy Jobs Plugin too, but that suffers from essentially the same problem: Jobs have to opt in to needing exclusive access, rather than by-default needing exclusive access. -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/eb839f37-fefc-4559-a84f-1709c2ceb25a%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
