On Mon, Apr 13, 2009 at 10:09 PM, Matt Wynne <m...@mattwynne.net> wrote:
> > On 13 Apr 2009, at 12:46, aslak hellesoy wrote: > > A couple of years ago I was on a project that had fallen into the trap of >> too many integration tests (exactly the horror scenario that J.B. >> Rainsberger describes: http://www.jbrains.ca/permalink/239). The whole >> suite had hundreds of slow watir tests and took several hours to run. If >> there was a failure, it was usually just in a coupel of them. >> >> We ended up improving this a lot with a homegrown distributed RSpec runner >> (based on Drb) and employing techniques from pairwise testing ( >> http://www.pairwise.org/). >> >> At the time I also explored a third technique that we never put in use: >> Heuristics. >> >> If we could establish relationships between arbitrary files and failing >> tests over time, then we would be able to calulate the probablilty that a >> certain commit would break certain tests. We could then choose to run the >> tests that had a high probablilty of breakage, and exclude the others. A >> neural network could potentially be used to implement this. The input >> neurons would be files in the codebase (on if it's changed, off if not) and >> the output neurons would be the tests to run. >> >> So if someone develops a better AutoTest with a plugin architecture, and >> that doesn't have to run as a long/lived process, then I'd be very >> interested in writing the neural network part - possibly backed by FANN ( >> http://leenissen.dk/fann/) >> >> It's so crazy it has to be tried! >> >> Aslak >> > > Aslak as usual you have stepped it up another seventy levels! > > I'm really keen to write a generic test runner that works on both RSpec and > Cucumber examples alike (and could extend for other testing libraries). > Allowing for pluggable strategies for test selection as Stephen has > described is a terrific idea. > > Aslak do you have any ideas in the pipeline for building a fancier test > runner for Cucumber? I know there was some brain-storming on the list a > while back about a 'thick client'. > Maybe ;-) Aslak > > > Matt Wynne > http://beta.songkick.com > http://blog.mattwynne.net > > > > _______________________________________________ > rspec-users mailing list > rspec-users@rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >
_______________________________________________ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users
