I have a similar issue. Among running pipelines we have also some matrix jobs. And allowing pipelines to even trigger requires allowing matrix parents on master. Matrix parents happen to clone SCM first and then they run sub-jobs. This can easily end up consuming all space on jenkins master. In my oppinion it is a bad approach forcing pipelines flyweight executor run on master.
I am not sure if this is bad approach of jenkins pipelines or matrix project plugin. On Tuesday, January 24, 2017 at 11:46:12 PM UTC+1, Spencer Malone wrote: > > We just encountered this very thing. Large repo with lots of PRs, threw in > a very simple Jenkinsfile and there was an organization folder set up. It > cloned a copy of the repo for each PR, since they all got triggered > together, and very quickly ran out of space. We had kept the disk space > relatively small on the master, since we didn't let it have any executors, > but now we know. Has anyone figured out a long term fix for this? > > On Wednesday, August 10, 2016 at 2:31:45 PM UTC-4, Kevin Phillips wrote: >> >> In my opinion, even more problematic than possibly running unsafe code in >> a flyweight executor on the master is the fact that these flyweight >> pipeline threads perform a full checkout of the target repository - at >> least when using Git as the SCM. So if one has a relatively large code >> repository containing product code - say several gigabytes - and they want >> to store their Jenkinsfile with the code so they can leverage the >> MultiBranch options in Jenkins Pipeline, they end up with many copies of >> their entire codebase being checked out on the master, possibly causing the >> master to run out of disk space. Add to this the fact that these >> "temporary" workspaces created on the master by the flyweight tasks are not >> automatically deleted when not in use and there is no way to purge them >> from the Jenkins dashboard and you are just asking for problems. This isn't >> a case of whether this will become a problem as much as it is an issue of >> when it will become a problem. >> >> In my opinion, Jenkinsfile's stored in SCM should be checked out on an >> agent as a typical freestyle job would do, and a default "node" allocated >> by the executor that launches the script contained therein for the node on >> which it is already running on. Then the rest of the Jenkinsfile could be >> processed in place without requiring any sort of flyweight executor at all, >> and avoiding the issues with having multiple checkouts of the same code >> spread across a build farm. Also, I think this would be inline with how >> some other more modern continuous integration servers work like travis-ci, >> so it's not without precidence. >> >> On Monday, June 13, 2016 at 6:03:20 PM UTC-3, Eli White wrote: >>> >>> We follow the Jenkins configuration best practices and have no executors >>> on our master node and force everything to run on our agents. >>> >>> We are starting to work with pipeline jobs and are worried that bad >>> Jenkinsfiles could cause problems on our master. >>> How can we force the flyweight jobs to run on a designated machine >>> *other* than master? >>> >>> How do other people handle this? >>> >> -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/86678208-2048-4d85-af89-21ae4dd9d0c2%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
