Replying to both of these emails. The issue with maven (like Raul points out) is that there are singletons. IE shared ports, files, etc.
With containers, this is removed entirely. Each container has its own set of ports. So going this route (using 24 containers) means a 24 hour build would finish in 1 hour. This way we could get MUCH faster iteration on releases and we could integrate every merge. Not sure we could integrate every commit.. My company integrates every commit but our integration is only 15 minutes. Also, this way we can use commodity machines. Otherwise you would have to buy 1-2 BIG boxes with 64GB of RAM and 16 cores. On Sun, Aug 9, 2015 at 5:14 PM, Christopher Shannon < christopher.l.shan...@gmail.com> wrote: > I've been thinking about how to improve the speed of builds as well and I > think the biggest thing is just getting the tests to run in parallel, > regardless of whether or not some form of continuous integration is used. > Maven supports parallel tests but it would be more work than just splitting > up the tests into more modules or running them in parallel. I imagine > there would be potential failures if all of the sudden we start running the > tests in parallel and that would need be addressed. > > > On Sun, Aug 9, 2015 at 3:43 PM, Kevin Burton <bur...@spinn3r.com> wrote: > > > Hey guys. > > > > Right now the ActiveMQ integration takes a long time. Last time we > > discussed this (not sure if it was on the list) it was about 24 hours. > > > > I’ve been playing with our internal builds and using CircleCI’s parallel > > integration and I reduced our builds from 50 minutes down to 15. > > > > I think 15 is as FAST as I can get it because there’s a constant > > compilation time of about 3-4 minutes … and then a dependency cache > restore > > takes another 3 minutes. > > > > I think this strategy could work for ActiveMQ too. > > > > What I did was I broke up large sets of tests that are in modules into > > sub-modules. > > > > So if I have say 150 tests in a module, I’ll split these into 5 modules > > each with 30 test. > > > > Then I run modules 0-1 on pipeline0, 1,2 on pipeline1, and so on. This > way > > my modules are split and can all test in parallel. > > > > My ‘prepare’ for each pipeline is to just “mvn install” the .jars so > when I > > test the modules they can don’t need any dependencies built. > > > > I think if we got a 24x pipeline, that we could get integrations down to > > 30-60 minutes. Maybe more if we went with higher parallelization. > > > > It wouldn’t be free though.. I think they give OSS project 3 free > > containers but we would require more than that. I suspect it would be > > about $1k per month. > > > > It won’t break compatibility with the traditional maven build. You can > > still run your maven build with modules. The only downside is that now > you > > have some dedicated modules with only have tests. > > > > Kevin > > > > -- > > > > Founder/CEO Spinn3r.com > > Location: *San Francisco, CA* > > blog: http://burtonator.wordpress.com > > … or check out my Google+ profile > > <https://plus.google.com/102718274791889610666/posts> > > > -- Founder/CEO Spinn3r.com Location: *San Francisco, CA* blog: http://burtonator.wordpress.com … or check out my Google+ profile <https://plus.google.com/102718274791889610666/posts>
