Re: [Puppet-dev] Order dependent test failures

2012-04-18 Thread Jeff McCune
On Tue, Apr 17, 2012 at 6:47 PM, Daniel Pittman wrote: > I just bisect with `rspec spec/[a-m]* spec/my/failing_spec.rb`, and > then narrow it down that way. The script is a faster way to do that. > > I never found debugger inspection very useful compared to bisection > because the source is often

Re: [Puppet-dev] Order dependent test failures

2012-04-17 Thread Daniel Pittman
I just bisect with `rspec spec/[a-m]* spec/my/failing_spec.rb`, and then narrow it down that way. The script is a faster way to do that. I never found debugger inspection very useful compared to bisection because the source is often very, very opaque and distant - so unless I knew what I was look

Re: [Puppet-dev] Order dependent test failures

2012-04-17 Thread Chris Price
Yeah, unfortunately, commenting out large swaths and then adding back in a little bit at a time is about the best weapon I've found to combat those situations so far. Would love to hear if anyone has found anything better. On Tue, Apr 17, 2012 at 6:41 PM, Jeff McCune wrote: > On Tue, Apr 17, 20

Re: [Puppet-dev] Order dependent test failures

2012-04-17 Thread Jeff McCune
On Tue, Apr 17, 2012 at 6:11 PM, Chris Price wrote: > Doh. I didn't click through to your gist before I replied. It looks like > you've already identified the problematic test, so most of what I posted > will be of no use to you. :( Well, I know my test works in isolation, but I don't know w

Re: [Puppet-dev] Order dependent test failures

2012-04-17 Thread Daniel Pittman
On Tue, Apr 17, 2012 at 17:27, Jeff McCune wrote: > I'm working on fixing up a module and I've written a test that triggers the > bug against Puppet 2.7.x.  I'm running into a problem where I get 0 failures > when I run `rake test` or `rspec --format d spec/` but I get the failure I'm > expecting

Re: [Puppet-dev] Order dependent test failures

2012-04-17 Thread Chris Price
Doh. I didn't click through to your gist before I replied. It looks like you've already identified the problematic test, so most of what I posted will be of no use to you. :( On Tue, Apr 17, 2012 at 5:27 PM, Jeff McCune wrote: > Hey everyone, > > I'm working on fixing up a module and I've wri

Re: [Puppet-dev] Order dependent test failures

2012-04-17 Thread Randall Hansen
On Apr 17, 2012, at 5:27 PM, Jeff McCune wrote: > Here's the gist of what's going on: 1,000 Internets for the correct use of "gist." :) r -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to puppet-dev@go

Re: [Puppet-dev] Order dependent test failures

2012-04-17 Thread Chris Price
Nick Lewis has a script that you can grab from here: https://gist.github.com/1217637 If you know the order that your tests ran it, this will help you narrow it down to the particular two that are interfering with one another (accomplished via a sort of binary search, running half of the number of

[Puppet-dev] Order dependent test failures

2012-04-17 Thread Jeff McCune
Hey everyone, I'm working on fixing up a module and I've written a test that triggers the bug against Puppet 2.7.x. I'm running into a problem where I get 0 failures when I run `rake test` or `rspec --format d spec/` but I get the failure I'm expecting if I run the spec file directly with rspec s