While I agree with the "your test suite should be fast or you're doing it wrong" crew to a certain extent, we've found any problem worth solving needs cucumber and integration tests anyway so parallelizing CI is a must regardless of how fast you can get your unit tests.
Like most people we've done parallel_tests but it has issues when you run both rspec and cucumber (or need to run across more than 1 box). We're currently using an in house gem called Nitra (look at github rather than rubygems: https://github.com/newporta/nitra) to parallelize our build across a bunch of commodity hardware and use Jenkins to manage the queuing and CI. That gives us easy access to running the full suite on our unmerged git branches using Jenkins' copy job feature. We've also tried Hydra but it had some fatal design flaws that kept us from using it in anger. It's in full time usage running a few dozen builds a day. It's based on decent unix conventions (abuses fork and pipe semantics for speed/memory reasons) and we're just about to get the upgrade work done to get it RSpec 2.12. compatible. Feel free to ask questions if you're interested, it's currently a pretty steep integration curve since we're the only ones currently using it and all the support stuff is buried in our ci rake tasks. On Monday, November 26, 2012 1:01:56 PM UTC+13, Sebastian Porto wrote: > > Adam, I agree on most of what you are saying, you have described many of > our problems with create accuracy (lots of factories, many, many candidates > for extraction) and we are certainly guilty of not running the full test > locally before sending to CI. It is really easy to end up in that place. We > have been looking into ways to split our app, but we haven't done any of > this yet. We'll get there. > > Sebastian > > Adam Boas <javascript:> > 26 November 2012 10:18 AM > I know this opinion is probably going to be unpopular, but I just want to > put out it there: > > If you need to parallellize your build, then your problems getting > parallel building working are certainly not your biggest problem. > > Even if you get a parallel build working, and it is certainly doable. > Where is that going to go? > > Step 1 people will stop doing a pre-commit build - The fact that you need > a parallel build means that it is almost certainly prohibitively time > consuming for a dev to do one. > > Step 2, people start caring less about build efficiency. Why care, you > can't run it locally anyway. The net effect of this is that your parallel > build will eventually start taking as long or longer than your old serial > one. > > Step 3 big refactorings become super scary, prohibitively time consuming > or simply not doable. This is because you can't quickly re-run your specs > anymore. A core refactor to be tested will have to go back and forth to the > build server to be verified (probably many times). That means you are going > to need another set of parallel builders focussing on another branch so > that you don't go screwing up the deployable master. > > Step 4, the super smart guy/gal who built your parallel build system > leaves the team and now you are in a world of pain :-) But seriously, a lot > of effort will go into understanding, maintaining and expanding your build. > > Every project I have worked on that required a parallel build had 3 common > threads: > > > 1. Lots of Factory based (database dependent) unit-level specs > 2. Large amounts of what should really be orthogonal, unit level specs > being done in acceptance tests > 3. A large application with more than one candidate application > extraction > > > I have been privileged to work with some super smart people, and the > projects suffering from the problems above were no exception, so it wasn't > that they were poor developers or bad designers. Generally early > extractions don't get done when there are tight deadlines. By the time you > are feeling that your build is too slow and it is starting to get in the > way of productivity, your application is already probably too big. At this > point I recommend trying to identify groups of functionality that are > candidates for an extraction. If they can operate completely independently > and be deployed completely independently then that is great. Worst case > scenario look to extract large pieces out as mountable engines. > > There are certainly non-trivial issues with maintaining multiple, > interdependent apps, but all-in-all I think teams will maintain greater > flexibility and development speed if effort is put into this kind of > modularisation rather than ever more complicated build structures designed > to stop the team needing to address the design issue. > > > Adam Boas > e: adam...@gmail.com <javascript:> > m: +61 457 741 117 > > > > > > -- > You received this message because you are subscribed to the Google Groups > "Ruby or Rails Oceania" group. > To post to this group, send email to rails-...@googlegroups.com<javascript:> > . > To unsubscribe from this group, send email to > rails-oceani...@googlegroups.com <javascript:>. > For more options, visit this group at > http://groups.google.com/group/rails-oceania?hl=en. > Thomas Egret <javascript:> > 26 November 2012 9:37 AM > I have been using parallel_tests as well and it's very awesome however > sometimes the tests suite just run smoothly and sometimes just fails so we > had to deactivate it for the moment (wonder if you use it as well on > Travis). > > Wish some people could give some hints regarding how to ensure my tests to > pass using parallel_tests. > > > > -- > You received this message because you are subscribed to the Google Groups > "Ruby or Rails Oceania" group. > To post to this group, send email to rails-...@googlegroups.com<javascript:> > . > To unsubscribe from this group, send email to > rails-oceani...@googlegroups.com <javascript:>. > For more options, visit this group at > http://groups.google.com/group/rails-oceania?hl=en. > James Healy <javascript:> > 26 November 2012 9:28 AM > A previous project I worked on used Jenkins with a vagrant plugin to run > concurrent tests in isolated VMs. > > It took a bit of setting up, but once it got going it was amazing and > dropped our build time from >20 minutes to <5 minutes. > > The downside was that it became tricker to understand the "state of the > build" at a glance. Instead of checking 1 project, you had to check 6 (unit > tests, acceptance tests 1, acceptance tests 2, etc). > > We also tried parallel_tests for a while and it mostly worked, but we ran > into a few issues with parallel acceptance specs stepping on each others > toes. > > James > > > > -- > You received this message because you are subscribed to the Google Groups > "Ruby or Rails Oceania" group. > To post to this group, send email to rails-...@googlegroups.com<javascript:> > . > To unsubscribe from this group, send email to > rails-oceani...@googlegroups.com <javascript:>. > For more options, visit this group at > http://groups.google.com/group/rails-oceania?hl=en. > Sebastian Porto <javascript:> > 22 November 2012 11:12 PM > Hi Guys > > There was a discussion the other day about CI server and it was mention > several times how good it is to run the tests in parallel. I am intrigued > by this and will like to implement this. But I don't know where to start. > > What is your approach for running CI test in parallel? We are using > TeamCity at the moment, were using Jenkins before. And Rspec. > > Thanks > Sebastian > -- > You received this message because you are subscribed to the Google Groups > "Ruby or Rails Oceania" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/rails-oceania/-/HqhReVKyNx0J. > To post to this group, send email to rails-...@googlegroups.com<javascript:> > . > To unsubscribe from this group, send email to > rails-oceani...@googlegroups.com <javascript:>. > For more options, visit this group at > http://groups.google.com/group/rails-oceania?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Ruby or Rails Oceania" group. To view this discussion on the web visit https://groups.google.com/d/msg/rails-oceania/-/q8fbR0mHz3sJ. To post to this group, send email to rails-oceania@googlegroups.com. To unsubscribe from this group, send email to rails-oceania+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rails-oceania?hl=en.
