|
Well certainly I'd be unwilling to give up multiple
assertions per unit test. Not all tests are so simple that there is
exactly one value being verified. When you're asserting multiple values
within a test you're usually doing one of the following:
1. Verifying the sanity of the test
fixture.
2. Verifying some compound state that involved several
different values.
3. Verifying the extended lifecycle by performing a
sequence of operations upon the object under test and verifying each
one.
Of these, the only case I'd be concerned about is #3 but in
practice it's not much of a problem. There is no way to verify the
lifecycle of the object under test without actually making it go through its
transitions. So you might as well have assertions after each transition to
check. You could decompose the test such that each transition is in its
own method and we chain them together but IMNSHO that's often a waste of
time. A compiler could perform this transformation for you if you really
care that much. Sure the test can become more complicated but that
shouldn't be surprising given its objective. As per usual the appropriate
response is to document the test more clearly, use sensible messages when
reporting assertion failures and try to keep the test focused on one
particular scenario.
Also note that not everyone uses MbUnit for unit
testing! Our Quality Engineering department uses a web testing framework
built on top of MbUnit. The one assertion per test rule would be
completely inappropriate for them.
I think this last point is important so I'm going to repeat
it again. MbUnit is used for more than just unit testing! The
framework should provide affordances for other kinds of testing
modalities. As hammer's go, it's a very nice one...
I've actually been thinking about ways the framework could
be opened up further to enable more intricate reporting scenarios. For
instance, our QE team would like to include periodic screenshots throughout the
execution of their web tests. There's no way to add this information to
the MbUnit report short of creating a custom test runner. Likewise, it
should be possible to integrate monitoring of performance counters and other
system resources and include them as out-of-band information in a
report.
I would see no harm in a multi-assert facility being built
that did something similar by recording the additional failures out-of-band to
be included in the report rather than throwing assertions.
Here's one clean way to do it. Create a block within
which you are creating a "composite" assertion. A composite assertion is
simply a block of code, all of whose assertions will be allowed to run because
AssertionExceptions will not be thrown. Their results will be logged
instead. At the end of the block, we check whether a failure was noted and
if so we raise a failure of our own. So composite assertions could be
nested.
DataStructure foo =
objectUnderTest.DoSomething();
using (Assert.Composite("Returned data structure contains
all expected values"))
{
Assert.AreEqual("5",
foo.X);
Assert.AreEqual(42,
foo.Y);
CollectionAssert.AreEqual(new int[] { 1,
2, 3, 4 }, foo.Z);
}
Another possibility is something like Assert.Negate()
though I'd probably avoid it. We could of course do
Assert.ExpectedException using an anonymous delegate.
For QE's reporting woes, I'd want to have some means of
recording additional information to include in reports. Could be used here
too. The basic idea is to make it possible for a test to append structured
information to the report.
Create something like an AssertionHandler to capture what
happens when an assertion passes or fails. The default when an assertion
fails is to throw an exception and do nothing when an assertion passes.
When inside a Composite assert block, we redirect assertion failure
handling to do something different. It could call a reporting method to
report the failure instead of throwing.
We could use those same reporting methods
too:
Report.Add(new AssertionFailureRecord("My assertion
failed."));
Report.Add(new AssertionPassRecord("My assertion
passed."));
And for our QE department, I could do something like this
deep in the guts of my web test framework:
Report.Add(new WebPageSnapshot("some html",
anImage));
A comparison assertion could record the exact strings
used rather than just some formatted representation:
public void AreEqual(object expectedValue, object
actualValue)
{
Report.Add(new
AssertionFailureRecord(String.Format(....), new EqualityAssertion(expectedValue,
actualValue));
}
Notice the power we gain here by being able to record
structured information in a report. We could use it to provide better
affordances in the test runner because there's extra meta-data in the report
regarding each assertion (and other things besides that)...
Jeff. From: [email protected] [mailto:[EMAIL PROTECTED] On Behalf Of Phil Haack Sent: Wednesday, October 04, 2006 9:45 AM To: [EMAIL PROTECTED]; [email protected] Subject: MbUnit Re: Mutiple asserts in a single fixture I think Roy Osherove
has a good argument against this. Phil From:
[EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Andrew Stopford Hi, I wanted to raise an idea from James Avery about
multiple asserts per fixture with addtional attribute and assert
markers. I have suggested to James that combintional test could
help where your looking for mutiple inputs, however Jamies idea is to take
several asserts in one fixture to let you conduct several tests. It's an
interesting idea but one I am not sure about in that mbunit like other
frameworks stops at the point of failure so that you fix that before re-running
your tests again and that makes sense to me. Jame's idea is that if you get a
fail then if the marker allows it you carry on but then fail the test when the
run completes (and report the fail(s). I'd welcome your thoughts on this, would
it be useful, does it make sense? Andy --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "MbUnit.User" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/MbUnitUser -~----------~----~----~----~------~----~------~--~--- |
- MbUnit Re: Mutiple asserts in a single fixture Jeff Brown
- MbUnit Re: Mutiple asserts in a single fixture Phil Haack
- MbUnit Re: Mutiple asserts in a single fixture Andrew Stopford
- MbUnit Re: Mutiple asserts in a single fixture Jeff Brown
