I'm not working at large scale, but there have been several presentations about large scale Jenkins which might offer ideas.
http://www.slideshare.net/kohsuke/large-scale-automation-with-jenkins or https://www.youtube.com/watch?v=8GAAL7T_ozM http://www.confreaks.com/videos/1213-jucsf2012-how-jenkins-builds-the-netflix-global-streaming-service https://www.cloudbees.com/sites/default/files/Jesse-Dowdle_Joel-Johnson_Masssively-CI.pdf https://www.youtube.com/watch?v=0D7PbIZLNSM http://www.cloudbees.com/jenkins-user-conference-2011-session-abstracts-video.cb (possibly the "Red Hat on Jenkins: By the Numbers" talk) Thanks, Mark Waite On Sun, Jun 22, 2014 at 6:37 PM, <[email protected]> wrote: > > On Sunday, June 22, 2014 4:05:40 PM UTC-7, Stephen Connolly wrote: >> >> It is a work in progress: https://groups.google.com/forum/m/#!topic/ >> jenkinsci-dev/qrG7bAnZSHQ >> >> Both that plugin and my literate multi-branch stuff are probably big game >> changers in how Jenkins will evolve. I suspect the workflow thing could be >> much bigger at the risk of being more complex.... It's the old "with great >> power comes great responsibility" thing. >> >> Certainly given its scriptability and fork-join capabilities I think it >> will be exactly what you need... Whereas I see literate eating the other >> end and simplifying less complex jobs for the 90% >> > > Oh no - from the looks of things, this plugin isn't available yet. > Definitely not good. You mentioned fork-join capabilities - I believe that > is precisely the kind of functionality we need. > > Using the example I posted above, would it be correct to say that we could > use this plugin on *Stage 2* to fork 281 different jobs to build our > libraries, wait for all those jobs to be finished by our 60 to 70 builder > cluster, join all 281 jobs back into the flow of execution, and then repeat > the process for *Stage 3*? > > If so, then it seems our adoption of Jenkins will hinge on the release of > this plugin. I read the link you posted above, Stephen. It looks like this > plugin was just announced last month, and has yet to reach a 0.1 release. > How were folks using Jenkins before May 13th (the date of the first message > in that thread) to build large software projects in an efficient > distributed fashion? Or was that just not possible? > > I have to believe it's possible, because I've heard of multiple > enterprise-level organizations using Jenkins to automate their build > processes over the years, but I can't see how they'd accomplish that > without the basic functionality this new plugin seems to provide. > > I'm very interested in your thoughts on this as well as the input of > others - is Jenkins not yet a production-ready technology, or is Jenkins > just not designed to handle this kind of build automation? > > Thank you Stephen, > -JJ > > -- > You received this message because you are subscribed to the Google Groups > "Jenkins Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- Thanks! Mark Waite -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
