Re: setup of Jenkins infrastructure
We have a setup with ~1000 matrix jobs building on about 50 platform / version combinations (i.e. one subjob is one platform combination). Total volume of the build logs approaches 1.5TB on a btrfs volume with compression enabled (I believe raw is about 2.5TB). If you were talking about freestyle jobs then our setup is way bigger than what you will have, otherwise its a similar size. Our experience has been that the Jenkins master becomes IOPS bound. You will really want a data storage system that can give you a high enough IOPS rate for the master. Otherwise the start up times are very long and the Jenkins webpage slows down badly. SSD is ideal if you can afford it, but only necessary for big setups with high amounts of jobs and build logs. Cheers Artur From: jenkinsci-users@googlegroups.com on behalf of LnT Sent: 01 March 2018 09:35 To: Jenkins Users Subject: Re: setup of Jenkins infrastructure Thanks so much Dirk. I will convince for Linux Jenkins master. Burden meaning - if 3000+ Jenkins jobs are created then even though jobs run @ slaves but respective build information will be stored @ Jenkins master. Hence it is seeing as burden. What would you suggest - if Jenkins page load is damn slow ? Regards, LnT On Thursday, March 1, 2018 at 12:43:51 PM UTC+5:30, Dirk Heinrichs wrote: Hi, > My Plan is > 1. Create Jenkins master in in windows VM (Provided 16GB RAM) We were also running our Jenkins master on Windows for the past couple of years, but we finally migrated it to Linux last year. Windows really (still) doesn't make a good server OS (even though it has "Server" in the name). So if there's no real reason other than "manager wants it to be this way", I'd recommend against running any server on Windows. > Cons : I suspect , 3000+ jobs will be created and heavy burden over Jenkins > master Please define "heavy burden". We usually don't run that much on the master, all the heavy lifting is done on our agent nodes. HTH... Dirk -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com<mailto:jenkinsci-users+unsubscr...@googlegroups.com>. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/6b3570ad-779f-4874-960f-a7ec7b3e1c8d%40googlegroups.com<https://groups.google.com/d/msgid/jenkinsci-users/6b3570ad-779f-4874-960f-a7ec7b3e1c8d%40googlegroups.com?utm_medium=email&utm_source=footer>. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/50011751a1bc4370bc7dc210ed442d33%40partner.eso.org. For more options, visit https://groups.google.com/d/optout.
Re: setup of Jenkins infrastructure
Very much grateful to your feedback Dirk. Thanks a lot. On Thursday, March 1, 2018 at 2:23:57 PM UTC+5:30, Dirk Heinrichs wrote: > Hi, > > > Burden meaning - if 3000+ Jenkins jobs are created then > > even though jobs run @ slaves > > but respective build information will be stored @ Jenkins master. Hence > it is seeing as burden. > > We've got approx. 1000 jobs, and don't experience any problems on the > master (VM with 4 vCPU and 16G RAM). We also use the "Archive Artifact" > plugin a lot, which stores to-be-kept artifacts on the master for later > retrieval. Some of those artifacts are sized ~3G. > > > What would you suggest - if Jenkins page load is damn slow ? > > A general mistake we made was to put all the jobs into the main page. It > was just lately that we discovered the wonders of the Folder plugin and > started to use it to organize our jobs. Views also help here, but I - > personally - like the folders more. > > Bye... > > Dirk > -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/3f692961-6562-4bcc-b3c8-836e1613a778%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: setup of Jenkins infrastructure
Hi, > Burden meaning - if 3000+ Jenkins jobs are created then > even though jobs run @ slaves > but respective build information will be stored @ Jenkins master. Hence it is > seeing as burden. We've got approx. 1000 jobs, and don't experience any problems on the master (VM with 4 vCPU and 16G RAM). We also use the "Archive Artifact" plugin a lot, which stores to-be-kept artifacts on the master for later retrieval. Some of those artifacts are sized ~3G. > What would you suggest - if Jenkins page load is damn slow ? A general mistake we made was to put all the jobs into the main page. It was just lately that we discovered the wonders of the Folder plugin and started to use it to organize our jobs. Views also help here, but I - personally - like the folders more. Bye... Dirk -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/BN7PR15MB24011BF311BC16E6FEC26004AAC60%40BN7PR15MB2401.namprd15.prod.outlook.com. For more options, visit https://groups.google.com/d/optout.
Re: setup of Jenkins infrastructure
Thanks so much Dirk. I will convince for Linux Jenkins master. Burden meaning - if 3000+ Jenkins jobs are created then even though jobs run @ slaves but respective build information will be stored @ Jenkins master. Hence it is seeing as burden. What would you suggest - if Jenkins page load is damn slow ? Regards, LnT On Thursday, March 1, 2018 at 12:43:51 PM UTC+5:30, Dirk Heinrichs wrote: > Hi, > > > My Plan is > > 1. Create Jenkins master in in windows VM (Provided 16GB RAM) > > We were also running our Jenkins master on Windows for the past couple of > years, but we finally migrated it to Linux last year. Windows really > (still) doesn't make a good server OS (even though it has "Server" in the > name). So if there's no real reason other than "manager wants it to be this > way", I'd recommend against running any server on Windows. > > > Cons : I suspect , 3000+ jobs will be created and heavy burden over > Jenkins master > > Please define "heavy burden". We usually don't run that much on the > master, all the heavy lifting is done on our agent nodes. > > HTH... > > Dirk > -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/6b3570ad-779f-4874-960f-a7ec7b3e1c8d%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: setup of Jenkins infrastructure
Hi, > My Plan is > 1. Create Jenkins master in in windows VM (Provided 16GB RAM) We were also running our Jenkins master on Windows for the past couple of years, but we finally migrated it to Linux last year. Windows really (still) doesn't make a good server OS (even though it has "Server" in the name). So if there's no real reason other than "manager wants it to be this way", I'd recommend against running any server on Windows. > Cons : I suspect , 3000+ jobs will be created and heavy burden over Jenkins > master Please define "heavy burden". We usually don't run that much on the master, all the heavy lifting is done on our agent nodes. HTH... Dirk -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/BN7PR15MB24016437B61667FB94C1385FAAC60%40BN7PR15MB2401.namprd15.prod.outlook.com. For more options, visit https://groups.google.com/d/optout.