On Mar 2, 2011, at 5:07 PM, Daniel Pittman wrote: > On Wed, Mar 2, 2011 at 16:32, Ben Hughes <b...@puppetlabs.com> wrote: >> On Wed, Mar 02, 2011 at 03:24:32PM -0800, Daniel Pittman wrote: >> >>> After every single test in the spec suite, assert that there are no >>> log messages at error or higher in the collected logs, before we flush >>> them. If there are, fail hard, and treat it as disaster. >> >> This looks a really good addition to the testing! >> Sure some tests will fail initially, but that's a good thing, right? >> >> Will ultimately fix more things than it breaks at the beginning. > > A few people have said this, so far, so: > > The branch is "work in progress" specifically because it still does > break a bunch of tests. That isn't something I am happy merging into > the main development line until those tests are fixed, or at least > identified to be genuine, important failures we didn't previously > detect. > > So, the R&D team would consider this entirely inappropriate to merge > until after we fixed that issue and had a clean build; this is > published for a sneak peek under the hood of what is going to land > later, in some form. > > Plus, I still hope someone can suggest a better way to check that > invariant after each test case completes.
I think your testing of the invariant is reasonable, but I think normal tests should probably use this: Puppet.expects(:warning) instead of testing that a given log is stored. That way the individual tests are a bit more isolated from the internals of how logs are stored. Really, you could almost say that the 'before' block should just have something like: Puppet.stubs(:warning).raises "Failed test" or something similar. This way you're entirely relying on the API. You probably want to actually use the internal Log class, rather than Puppet (since classes that include the Logging module don't use Puppet.whatever, I think). -- Measure with a micrometer. Mark with chalk. Cut with an axe. --------------------------------------------------------------------- Luke Kanies -|- http://puppetlabs.com -|- +1(615)594-8199 -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to puppet-dev@googlegroups.com. To unsubscribe from this group, send email to puppet-dev+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en.
