|
The main issue is the following, using the
example you gave: Assert.AreEqual("5", foo.X); Assert.AreEqual(42, foo.Y); Suppose that foo.Y depends on the value of
foo.X. The first assertion failing might make the second one pass, when it
shouldn’t.
Phil From:
[EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Jeff Brown Well certainly I'd be unwilling to give up
multiple assertions per unit test. Not all tests are so simple that there
is exactly one value being verified. When you're asserting multiple
values within a test you're usually doing one of the following: 1. Verifying the sanity of the test
fixture. 2. Verifying some compound state that
involved several different values. 3. Verifying the extended lifecycle by
performing a sequence of operations upon the object under test and
verifying each one. Of these, the only case I'd be concerned
about is #3 but in practice it's not much of a problem. There is no way
to verify the lifecycle of the object under test without actually making it go
through its transitions. So you might as well have assertions after each
transition to check. You could decompose the test such that each
transition is in its own method and we chain them together but IMNSHO that's
often a waste of time. A compiler could perform this transformation for
you if you really care that much. Sure the test can become more
complicated but that shouldn't be surprising given its objective. As per
usual the appropriate response is to document the test more clearly, use
sensible messages when reporting assertion failures and try to keep the
test focused on one particular scenario. Also note that not everyone uses
MbUnit for unit testing! Our Quality Engineering department uses a web
testing framework built on top of MbUnit. The one assertion per test rule
would be completely inappropriate for them. I think this last point is important so
I'm going to repeat it again. MbUnit is used for more than just unit
testing! The framework should provide affordances for other kinds of
testing modalities. As hammer's go, it's a very nice one... I've actually been thinking about ways the
framework could be opened up further to enable more intricate reporting
scenarios. For instance, our QE team would like to include periodic
screenshots throughout the execution of their web tests. There's no way
to add this information to the MbUnit report short of creating a custom test
runner. Likewise, it should be possible to integrate monitoring of
performance counters and other system resources and include them as out-of-band
information in a report. I would see no harm in a multi-assert
facility being built that did something similar by recording the additional
failures out-of-band to be included in the report rather than throwing
assertions. Here's one clean way to do it.
Create a block within which you are creating a "composite"
assertion. A composite assertion is simply a block of code, all of whose
assertions will be allowed to run because AssertionExceptions will not be
thrown. Their results will be logged instead. At the end of the
block, we check whether a failure was noted and if so we raise a failure of our
own. So composite assertions could be nested. DataStructure foo =
objectUnderTest.DoSomething(); using (Assert.Composite("Returned
data structure contains all expected values")) {
Assert.AreEqual("5", foo.X); Assert.AreEqual(42,
foo.Y);
CollectionAssert.AreEqual(new int[] { 1, 2, 3, 4 }, foo.Z); } Another possibility is something like
Assert.Negate() though I'd probably avoid it. We could of course do
Assert.ExpectedException using an anonymous delegate. For QE's reporting woes, I'd want to have
some means of recording additional information to include in reports.
Could be used here too. The basic idea is to make it possible for a test
to append structured information to the report. Create something like an AssertionHandler
to capture what happens when an assertion passes or fails. The default
when an assertion fails is to throw an exception and do nothing when an
assertion passes. When inside a Composite assert block, we redirect
assertion failure handling to do something different. It could call a
reporting method to report the failure instead of throwing. We could use those same reporting methods
too: Report.Add(new
AssertionFailureRecord("My assertion failed.")); Report.Add(new
AssertionPassRecord("My assertion passed.")); And for our QE department, I could do
something like this deep in the guts of my web test framework: Report.Add(new WebPageSnapshot("some
html", anImage)); A comparison assertion could record
the exact strings used rather than just some formatted representation: public void AreEqual(object expectedValue,
object actualValue) { Report.Add(new
AssertionFailureRecord(String.Format(....), new
EqualityAssertion(expectedValue, actualValue)); } Notice the power we gain here by being
able to record structured information in a report. We could use it to
provide better affordances in the test runner because there's extra meta-data
in the report regarding each assertion (and other things besides that)... Jeff. From:
[email protected] [mailto:[EMAIL PROTECTED] On Behalf Of Phil Haack I think Roy Osherove has a good argument
against this. Phil From:
[EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Andrew Stopford Hi, I wanted to raise an idea from James Avery about multiple asserts per
fixture with addtional attribute and assert markers. I have suggested to James that combintional test could
help where your looking for mutiple inputs, however Jamies idea is to take
several asserts in one fixture to let you conduct several tests. It's an
interesting idea but one I am not sure about in that mbunit like other
frameworks stops at the point of failure so that you fix that before re-running
your tests again and that makes sense to me. Jame's idea is that if you get a
fail then if the marker allows it you carry on but then fail the test when the
run completes (and report the fail(s). I'd welcome your thoughts on this, would
it be useful, does it make sense? Andy
--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "MbUnit.User" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/MbUnitUser -~----------~----~----~----~------~----~------~--~--- |
- MbUnit Re: Mutiple asserts in a single fixture Jeff Brown
- MbUnit Re: Mutiple asserts in a single fixture Phil Haack
- MbUnit Re: Mutiple asserts in a single fixture Andrew Stopford
- MbUnit Re: Mutiple asserts in a single fixture Jeff Brown
